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1.0  INTRODUCTION 


A  detailed  analysis  of  the  Air  Force  on-the-job  training  (OJT) 
system  has  explicitly  recognized  "...a  need  for  better  and  more  easily 
applied  methods  of  monitoring  and  evaluating  the  Air  Force  OJT  program 
in  terms  of  costs,  quality  and  effectiveness  of  training  provided...."! 
Moreover,  the  Air  Force  Inspector  General,  in  a  recently  completed 
Functional.  Management  Inspection  of  the  Air  Force  OJT  program,  has 
concluded  that  lack  of  cost  and  capacity  information  is  a  major  hind¬ 
rance  to  the  efficient  management  of  the  program.  Yet,  estimation  of 
the  capacity  of  Air  Force  operational  units  to  conduct  OJT  has  received 
little  attention  prior  to  this  report.  Therefore,  the  fundamental 
objective  of  this  study  is  to  develop  and  demonstrate  a  practical 
methodology  by  which  the  capacity  of  Air  Force  units  to  perform  OJT 
v  7  estimated. 

Mounting  pressures  to  obtain  the  most  cost-effective  mix  of  resi¬ 
dent  school  training  and  OJT  make  it  increasingly  important  for  the 
Air  Force  to  be  able  to  determine  the  OJT  capacities  of  its  opera¬ 
tional  units.  At  its  present  level  of  operation,  OJT  may  be  less 
costly  or  otherwise  more  efficient  than  resident  school  training  for 
some  AFSC  (Air  Force  Specialty  Code)  skill  categories  in  some  types 
of  Air  Force  units.  Yet,  it  would  be  unreasonable  to  replace  all 
resident  school  training  with  OJT  in  such  situations  if  doing  so  would 
cause  the  unit's  OJT  load  to  exceed  capacity.  At  the  extreme,  exces¬ 
sive  OJT  might  degrade  primary  mission  performance  below  acceptable 
standards.  More  typically',  as  training  loads  increase,  OJT  costs  are 
likely  to  rise  until  the  cost-effectiveness  of  OJT  relative  to  resident 
school  training  has  been  eliminated. 

Clearly,  Air  Force  decision-making  concerning  all  of  these  issues 
would  be  enhanced  by  a  workable  methodology  to  estimate  the  capacity 
of  operational  units  to  conduct  OJT.  This  report  presents  the  results 
of  an  initial  effort  to  develop  and  demonstrate  such  a  methodology. 

Thus,  Section  2.0  provides  background  material  concerning  the 
nature  of  OJT  and  summarizes  pertinent  elements  of  the  OJT  literature. 

A  conceptual  model  for  determining  OJT  capacity  is  then  developed  in 
Section  3.0.  Next,  Section  4.0  describes  the  selection  of  operational 
units  as  bases  for  empirically  testing  the  model  and  discusses  the 
data  sources  utilized  for  these  empirical  tests.  The  results  of  esti¬ 
mating  the  structural  equations  of  the  conceptual  model  of  OJT  capacity 


^Stephenson,  R.  W. ,  and  J.  R.  Burkett,  On-the-Job  Training  in  the 
Air  Force:  A  Systems  Analysis,  AFHRL-TR-75-83 ,  AD-A036  206,  Lowry  AFB, 
Colorado,  Air  Force  Human  Resources  Laboratory,  Technical  Training 
Division,  December  1975,  p.  9. 


for  the  selected  operational  units  are  reported  in  Section  5.0. 

Finally,  Section  6.0  summarizes  the  conclusions  derived  in  the  study 
and  presents  specific  recommendations  for  further  development  and 
implementation  of  the  OJT  capacity  estimation  methodology. 

The  conceptual  model  of  OJT  capacity  developed  in  this  study  is 
a  conqice  description  of  the  complex  interrelationships  among  the 
level  of  mission  performance,  the  quality  of  mission  performance,  the 
amount  of  training  provided,  and  the  quality  of  training  achieved  by 
an  Air  Force  operational  unit.  Thus,  the  model  incorporates  represen¬ 
tations  of  the  interdependence  among  these  four  attributes,  the 
resources  available  to  the  unit,  and  pertinent  external  factors.  The 
maximum  capacity  of  a  unit  to  conduct  OJT  is  then  determined  as  the 
maximum  amount  of  training  sustainable  by  the  unit  without  compromising 
established  standards  for  the  level  of  mission  performance,  the  quality 
of  mission  performance,  and  the  quality  of  training.  Similarly,  the 
desirable  OJT  capacity  of  a  unit  is  calculated  as  the  maximum  amount 
of  training  that  the  unit  can  conduct  while  maintaining  a  desirable 
balance  among  the  level  and  quality  of  mission  performance  and  train¬ 
ing. 


The  model  has  been  empirically  tested  by  applying  multiple  linear 
regression  analysis  to  estimate  the  basic  structural  equations  of  the 
model,  to  the  extent  possible,  using  existing  data  for  six  Air  Force 
operational  units.  The  operational  units  examined  include  four  air¬ 
craft  maintenance  units  responsible  for  servicing  aircraft  assigned 
to  primary  tactical  flying  missions,  one  transportation  squadron 
responsible  primarily  for  vehicle  maintenance,  and  one  supply  squadron. 
In  all  cases,  the  data  used  in  the  analyses  consist  of  measures 
routinely  developed  and  maintained  for  the  units  by  various  Air  Force 
offices. 

Despite  the  substantial  limitations  in  the  availability  of  suit¬ 
able  data,  the  empirical  analyses  have  revealed  a  persistent  and  strong 
inverse  relationship  between  the  amount  of  training  conducted  by  a  unit 
and  the  quality  of  mission  performance  attributable  to  the  unit  for  all 
operational  units  studied.  In  addition,  a  perceptible,  but  weaker, 
correlation  between  the  resources  available  to  a  unit  and  the  quality 
of  the  unit's  mission  performance  has  also  been  detected.  Finally, 
the  possibility  that  an  association  exists  among  the  quality  of  train¬ 
ing  provided  in  a  unit,  the  training  load  of  the  unit,  and  conceivably, 
the  resources  available  to  the  unit  is  also  suggested  by  the  empirical 
results.  However,  at  least  in  the  relatively  low  stress,  peacetime 
situations  examined  in  the  study,  no  consistent  interdependence  is 
discernible  among  the  level  of  mission  performance  achieved  by  a  unit , 
the  amount  of  training  conducted  by  the  unit,  and  the  availability  of 
resources  to  the  unit. 

The  derivation  of  numerous  statistically  significant  relationships, 
in  spite  of  the  relatively  poor  quality  of  the  data  obtainable  from 
secondary  sources  ar.d  the  absence  of  any  indicators  of  some  important 
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explanatory  variables  in  those  sources,  is  very  encouraging.  This 
favorable  result  suggests  that  further  development  of  the  conceptual 
model  of  OJT  capacity  formulated  and  preliminarily  tested  in  this 


study  can  reasonably  be  expects  i  to  produce  very  useful  methodologies 
for  estimating  OJT  capacity. 

Specifically,  the  derivation  of  an  implementable  OJT  capacity 
estimation  methodology  will  require  expanded  exploratory  research, 
supported  by  a  methodical  program  of  data  acquisition  and  retention 
designed  to  provide  a  comprehensive  set  of  indicators  describing  all 
of  the  variables  contained  in  the  conceptual  model.  Thus,  data  per¬ 
tinent  to  the  construction  of  indicators  of  each  variable  in  the 
conceptual  model,  possibly  compiled  at  the  level  of  individual  work 
centers,  should  be  collected  and  maintained  for  a  representative 
sample  of  Air  Force  operational  units.  Empirical  estimates  of  the 
coefficients  of  the  model's  basic  structural  equations  should  then 
be  derived  for  each  unit  in  the  sample  using  these  data.  Next,  the 
empirical  results  obtained  for  presumably  similar  units  should  be 
systematically  compared  and,  to  the  extent  possible,  reconciled. 
Quantitative  assessments  of  the  OJT  capacities  of  the  units  should 
then  be  produced  on  the  basis  of  the  reconciled  coefficient  estimates 
using  the  procedures  formulated,  but  not  applied,  in  this  study. 
Finally,  aggregation  procedures  should  be  developed,  and  empirically 
tested,  to  permit  the  calculation  of  OJT  capacities  for  higher  organi¬ 
zational  levels  than  squadrons  or  flying  wings. 

Detailed  discussions  of  the  conceptual  model,  the  data  bases, 
the  analytic  results ,  and  the  recommendations  for  further  development 
are  presented  in  the  remainder  of  this  report. 


2 . 0  BACKGROUND 


2.1  The  Nature  of  On-the-Job  Training 


Perhaps  the  most  important  reason  why  no  technique  has  yet  been 
developed  to  estimate  OJT  capacity  lies  in  the  sheer  size  and  diversity 
of  Air  Force  OJT.  .It  has  been  estimated  that,  at  any  given  time, 
approximately  10  percent  of  the  authorized  enlisted  strength  of  the 
Air  Force  is  engaged  in  the  program  in  some  capacity. 1  Virtually  all 
enlisted  personnel  ate  trained  on  the  job  to  some  extent.  In  addition, 
most  personnel  who  attain  at  least  the  5-skill  level  conduct  OJT  from 
time  to  time.  Consequently,  the  OJT  program  embraces  all  the  varia¬ 
tions  of  the  Air  Force  skill  spectrum,  and  embodies  all  the  peculiar¬ 
ities  of  individual  trainers  and  trainees. 

This  structural  complexity  is  further  confounded  by  the  absence 
of  a  definition  of  OJT  that  is  uniformly  accepted  throughout  the  Air 
Force.  Air  Force  Management  Engineering  Policies  and  Procedures 
describes  OJT  as: 

...a  productive  indirect  category  to  accommodate  time 
expended  by  a  worker  in  a  directly  supervised,  on-the- 
job  proficiency  training  status,  where  the  worker  is 
being  advised  and/or  assisted  by  the  supervisor,  but 
is  not  achieving  any  production.  If  a  worker  is 
"learning  while  producing,"  his  time  is  recorded  in 
the  appropriate  productive  category,  whether  it  be 
direct  or  indirect,  and  not  as  OJT. 2 

In  contrast,  AFM  50-23  states  that: 

On-the-job  training  is  an  all-inclusive  terra  which 
describes  any  and  all  training  received  by  an  airman 
while  he  is  performing  in  a  duty  assignment  of  his 
AFS  [Air  Force  Specialty]. ^ 


Stephenson,  R.  W, ,  and  J.  R.  Burkett,  On-the-Job  Training  in  the 
Air  Force:  A  Systems  Analysis,  AFHRL-TR-75-83,  AD-A036  206,  Lowry  AFB , 
Colorado,  Air  Force  Human  Resources  Laboratory,  Technical  Training 
Division,  December  L975,  p.  9. 

"^Department  of  the  Air  Force,  Management  Engineering  Policies  and 
Procedures ,  AFM  25-5,  Washington,  D.C.  ,  August  8,  1973,  plus  changes, 
p.  4-12. 

^Department  of  the  Air  Force,  On-the-Job  Training,  AFM  50-23, 
Washington,  D.C.  ,  August  15,  1974,  p.  1-1. 
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This  ambiguity  about  the  nature  of  OJT  is  reflected  in  the  estab¬ 
lished  procedures  for  the  maintenance  of  personnel  records.  Upgrade 
training  (UGT)  is  accorded  formal  OJT  status.  Personnel  are  explicitly 
enrolled  in  UGT;  and  training  records  are  maintained  throughout  their 
enrollment.  Conversely,  qualification  training  seldom  is  formally 
recognized  while  it  is  being  performed.  Rather,  such  training  typi¬ 
cally  is  recorded  in  a  trainee's  Job  Proficiency  Guide  (JPG)  only 
after  it  has  been  completed. 

Yet,  in  the  context  of  an  initial  research  effort  to  develop  a 
methodology  for  estimating  the  capacity  of  operational  units  to  per¬ 
form  OJT,  th  so  distinctions  among  the  various  types  of  training 
conducted  by  operational  Air  Force  units  are  of  minor  importance. 

All  such  training  imposes  similar  demands  upon  unit  resources. 

•  Qualified  instructors  and  appropriate  equipment  must 
be  provided  for  "hands-on"  proficiency  training. 

•  On  occasion,  equipment  essential  to  primary  mission 
performance  must  be  released  for  use  in  training. 

•  Work  schedules  must  be  arranged  to  bring  trainees, 
experienced  personnel,  and  suitable  equipment  together. 

•  For  certain  specialized  skills,  funds  must  be  budgeted 
for  trainees'  participation  in  field  training  detach¬ 
ment  (FTD)  programs. 

Thus,  regardless  of  the  precise  form  of  training  provided,  the  capacity 
of  an  operational  unit  to  perform  OJT  is  directly  dependent  upon  the 
availability  of  training  opportunities.  Moreover,  these  training 
opportunities,  in  turn,  will  depend  on  the  availability  of  unit 
resources  in  excess  of  those  required  for  primary  mission  performance. 

Consequently,  the  provision  of  OJT  and  primary  mission  performance 
will  typically  be  interdependent,  largely  through  their  competition  for 
available  resources.  In  addition,  this  interdependence  generally  will 
involve  more  than  just  an  interaction  between  the  level  of  mission 
performance  achieved  and  the  amount  of  OJT  conducted.  Rather,  both 
the  quality  of  mission  performance  attained  and  the  quality  of  train¬ 
ing  provided  usually  will  also  be  affected.  Thus,  inherent  in  the 
nature  of  OJT  is  a  complex  interrelationship  among  the  amount  of 
training  performed,  the  quality  of  training  provided,  the  level  of 
mission  performance  achieved,  and  the  quality  of  mission  performance 
attained;  and  the  development  of  a  methodology  to  estimate  OJT  capacity 
unavoidably  involves  analysis  of  this  interrelationship. 
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2.2  Review  of  Pertinent  Literature 


No  previous  study  has  directly  examined  the  concept,  of  the  capac¬ 
ity  of  operational  units  to  perform  OJT.  Numerous  studies  of  other 
aspects  of  OJT  have  been  conducted,  however.  Many  of  these  studies 
provide  useful  insights  into  the  measurement  of  OJT  capacity. 

The  studies  pertinent  to  the  estimation  of  OJT  capacity  all 
address  various  elements  of  the  cost-effectiveness  of  OJT  programs. 
Thus,  all  of  the  pertinent  studies  can  be  related  in  some  way  to  the 
basic  concept  that  the  cost-effectiveness  of  an  OJT  program  can  be 
evaluated  by  comparing  the  value  added  to  the  skills  of  the  program's 
trainees  with  the  costs  of  the  resources  used  in  the  program. 

Most  of  these  studies  have,  either  explicitly  or  implicitly,  used 
Increases  in  trainee  productivity  as  a  measure  of  the  value  added  by 
OJT,  and  have  employed  decreases  in  trainer  productivity  as  a  component 
of  training  costs.  These  valuation  techniques  have  been  applied  most 
directly  by  Gay4  and  by  Gay  and  Nelson^  who  have  estimated  the  net 
value  added  by  OJT  as  the  present  value  of  the  differences  over  time 
between  the  actual  productivity  of  trainees  in  the  specialties  in 
which  they  have  been  trained  and  the  productivity  of  these  trainees 
in  their  highest  valued  alternative  military  assignment. 

Productivity  differences  have  been  used  somewhat  less  directly 
by  Weiher  and  Horowitz^  and  by  O'Neill?  to  measure  OJT  costs.  Weiher 
and  Horowitz  include  as  an  element  of  their  cost  estimates  the  value 
of  foregone  trainee  output,  which  they  measure  as  the  difference  over 
time  between  the  trainees'  wages  and  the  value  of  their  direct  output. 
Similarly,  O'Neill  measures  OJT  costs  in  terms  of  the  productivity 
foregone  by  replacing  a  journeyman  with  a  trainee. 


Gay,  R.  M. ,  Estimating  the  Cost  of  On-the-Job  Training  in  Mili¬ 
tary  Occupations:  A  Methodology  and  Pilot  Study,  R-1351-ARPA,  Santa 
Monica,  California,  The  RAND  Corporation,  April  1974. 

^Gay,  R.  M. ,  and  G.  R.  Nelson,  Cost  and  Efficiency  in  Military 
Specialty  Training,  Santa  Monica,  California,  The  RAND  Corporation, 
January  1974. 

Weiher,  R. ,  and  S.  A.  Horowitz,  Formal  and  On-the-Job  Training 
for  Navy  Enlisted  Occupations,  Professional  Paper  No.  83,  Arlington, 
Virginia,  Center  for  Naval  Analyses,  November  1971. 

^O'Neill,  D.  M. ,  "Determinants  of  Labor  Turnover  Costs  in  the 


Military,"  Study  4,  Part  1,  Volume  I  of  Studies  Prepared  for  the 
President's  Commission  on  an  All-Volunteer  Armed  Force,  Washington, 
D.C. ,  U.  S.  Government  Printing  Office,  November  1970. 
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Finally,  Arzigian®  strongly  implies  the  use  of  productivity 
changes  as  a  measure  of  the  value  added  by  OJT  when  he  incorporates 
in  his  estimates  of  OJT  costs  the  assumption  that  a  trainee's  effec¬ 
tiveness  increases  linearly  over  time  from  the  conclusion  of  basic 
training  through  completion  of  OJT  and  attainment  of  the  journeyman 
level.  Thus,  in  combination,  all  of  the  studies  discussed  above 
clearly  indicate  both  a  direct  relationship  between  the  current  pro¬ 
vision  of  OJT  and  future  mission  performance,  and  an  inverse  correla¬ 
tion  between  current  OJT  activities  and  current  mission  performance. 
These  research  results  strongly  support  the  conclusion  reached  in 
the  preceding  section  that  developing  a  methodology  to  estimate  OJT 
capacity  will  necessarily  involve  examination  of  the  complex  inter¬ 
relationships  between  mission  performance  and  the  provision  of  OJT. 

Further  evidence  of  the  competitive  relationship  between  current 
OJT  efforts  and  current  mission  performance  is  provided  by  several 
studies  that  .estimate  the  cost  of  OJT  in  terms  of  the  value  of  the 
resources  devoted  to  the  provision  of  OJT.  These  studies,  including 
research  conducted  by  Dunham^  and  by  Eisele,  Bell  and  Laidlaw.10  as 
well  as  the  previously  cited  work  by  Arzigian^  and  by  Weiher  and 
Horowitz, 6  all  identify  the  value  of  trainers'  and  supervisors'  time 
devoted  to  OJT  as  key  components  of  OJT  cost.  In  the  absence  of 
corresponding  increases  in  overtime,  allocation  of  these  resources 
to  OJT  will  necessarily  reduce  the  amount  of  experienced  personnel 
effort  available  for  primary  mission  performance.  Thus,  at  the 
extreme,  the  excessive  allocation  of  skilled  personnel  resources  to 
conduct  OJT  can  cause  mission  performance  to  decline  below  acceptable 
standards  and  thereby  cause  OJT  capacity  to  be  exceeded. 

Finally,  a  study  by  Lecznar^ 1  has  examined  the  relative  effective¬ 
ness  of  OJT  and  formal  resident  school  training  in  terms  of  six  criteria 
related  to  the  primary  mission  performance  of  graduates  of  the  two  types 


g 

Arzigian,  S. ,  On-the-Job  Training  Costs:  An  Analysis,  U.  S.  Navy, 
Bureau  of  Naval  Personnel,  WRM  67-52,  Washington,  D.C.,  June  1967. 

9 

Dunham,  A.  D. ,  Estimated  Cost  of  On-the-Job  Training  to  the  3-Skill 
Level  in  the  Communications  Center  Operations  Specialty,  AFHRL-TR-72-56, 
AD-753  093,  Lackland  AFB,  Texas,  Air  Force  Hitter  Resources  Laboratory, 
Personnel  Research  Division,  June  1972. 

^Eisele,  C,  R. ,  T.  R.  Bell,  and  C.  D.  Laidlaw,  Cost  Analysis  of  Air 
Force  On-the-Job  Training:  Development  and  Demonstration  of  a  Method¬ 
ology,  AFHRL-TR-78-88,  Lowry  AFB,  Colorado,  Air  Force  Human  Resources 
Laboratory,  Technical  Training  Division,  May  1979. 

iLecznar,  W.  B. ,  The  Road  to  Work;  Technical  School  Training  or 
Directed  Duty  Assignment,  AFHRL-TR-72-29 ,  AD-754  845,  Lackland  AFB, 

Texas,  Air  Force  Human  Resources  Laboratory,  Personnel  Research  Division, 
April  1972. 
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of  training.  Specifically,  the  mission  performance  of  technical  school 
and  OJT  graduates  are  compared  in  terms  of  job  difficulty,  average  task 
difficulty,  number  of  tasks  performed,  job  interest,  utilization  of 
talent  and  training,  and  overall  job  performance.  This  study,  which 
reveals  no  significant  differences  between  OJT  and  formal  resident 
school  training,  suggests  that  measures  of  post-training  performance 
represent  potentially  useful  indicators  of  training  quality.  This 
insight  has  direct  relevance  to  the  formulation  of  quantitative  indi¬ 
cators  of  training  quality  for  use  in  empirically  investigating  the 
estimation  of  OJT  capacity. 


3.0  CONCEPTUAL  MODEL  OF  OJT  CAPACITY 


Based  on  Che  review  of  available  literature  and  the.  initial  explor¬ 
ations  into  the  nature  of  OJT,  four  key  concepts  have  been  identified 
as  the  basic  elements  of  a  conceptual  model  of  OJT  capacity.  These 
concepts  are 

«  The  level  of  mission  performance. 

•  The  quality  of  mission  performance. 

•  The  quality  of  training  provided. 

•  The  amount  of  training  provided. 

The  predominant  concern  of  any  Air  Force  operational  unit  is  the 
successful  performance  of  its  primary  mission.  Accordingly,  any  degra¬ 
dation  in  the  level  or  quality  of  mission  performance  below  acceptable 
standards  cannot  be  tolerated.  Training  loads  causing  such  degradation 
would  be  considered  to  exceed  capacity. 

Similarly,  the  amount  of  training  provided  is  meaningless  if  its 
quality  is  inadequate.  Therefore,  levels  of  training  that  cause  train¬ 
ing  quality  to  deteriorate  below  some  minimum  standard  would  also  be 
considered  to  exceed  capacity. 

These  basic  relationships  are  depicted  graphically  in  Figure  1. 

Cl,  C2 ,  and  C3  represent  the  maximum  amounts  of  training  sustainable 
without  compromising  the  established  standards  for  the  level  of  mission 
performance,  the  quality  of  mission  performance,  ar.d  the  quality  of 
training,  respectively.  Thus,  the  OJT  capacity  indicated  in  this 
figure  is  C2.  This  amount  of  training  is  the  maximum  training  load 
for  which  the  level  of  mission  performance,  the  quality  of  mission 
performance,  and  the  quality  of  training  all  fulfill  their  minimum 
standards.  Any  greater  amount  of  training  would  cause  the  quality 
of  mission  performance  to  decline  to  an  unacceptable  level. 

This  representation  considerably  oversimplifies  the  determination 
of  OJT  capacity,  however.  The  level  of  mission  performance,  the  quality 
of  mission  performance,  and  the  quality  of  training  are  unlikely  to  be 
independent.  Rather,  training  quality  and  mission  performance  quality 
both  can  reasonably  be  expected  to  decline  as  the  level  of  mission  per¬ 
formance  increases,  at  least  at  higher  mission  performance  levels. 
Moreover,  training  quality  and  mission  performance  quality  are  also 
likely  to  be  interrelated,  although  the  nature  of  the  correlation  is 
more  complex.  Current  mission  performance  quality  should  be  positively 
influenced  by  the  quality  of  previous  training  insofar  as  training 
improves  productivity.  Conversely,  current  training  quality  and 
current  mission  performance  quality  may  be  negatively  related,  since 
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Level  of  Mission  Performance 


FIGURE  It  Conceptual  Model  of  OJT  Capacity 


their  provision  draws  upon  a  common  pool  of  resources.  For  example, 
skilled  personnel  performing  training  duty  might  improve  training 
quality;  while  their  absence  from  directly  productive  activities 
might  reduce  the  quality  of  mission  performance. 

In  addition,  other  factors,  such  as  resource  availability,  job 
characteristics,  organizational  considerations,  and  external  influences 
can  also  affect  the  level  and  quality  of  both  mission  performance  and 
training.  A  more  detailed  list  of  such  factors  is  presented  in  Table  1. 

All  of  the  propositions  discussed  should  be  incorporated  in  any 
comprehensive,  operational  model  of  OJT  capacity.  These  propositions, 
and  their  interrelationships,  can  be  conveniently  summarized  mathemat¬ 
ically  in  terms  of  the  following  system  of  equations: 

ML  -  f  (TL,  TQp,  R,  E)  (1) 

MQ  “  g  (ML,  TL,  TQ,  TQ  ,  R,  E) 

P 

TQ  =  h  (ML,  TL,  R,  E) 

where  ML  *  the  level  of  mission  performance. 

MQ  *  the  quality  of  mission  performance. 

TL  *  the  amount  of  training  provided  (the  training 

load  whose  maximum  attainable  value  defines 
the  OJT  capacity  of  the  operational  unit). 

TQ  •  the  quality  of  current  training. 

TQp  *  the  quality  of  previous  training. 

R  =  the  resources  available  to  the  operational  unit. 

E  *  external,  and  other,  factors. 

The  analysis  presented  previously  in  this  section  has  examined  in  con¬ 
siderable  detail  the  anticipated  nature  of  the  interrelationships  among 
the  variables  in  these  equations.  These  anticipated  interrelationships 
can  be  restated  conveniently  in  terms  of  the  expected  signs  of  the 
first-order  partial  derivatives  of  the  equations.  Specifically,  the 
anticipated  signs  of  those  partial  derivatives  for  which  unequivocal 
expectations  can  be  formed  are 

•  For  equation  (1),  dML/dTL < 0,  dML/dTQ  >0,  and  dML/dR  >0. 

P 

•  For  equation  (2),  dMQ/dML  <  0 ,  dMQ/dTL  <0 ,  5MQ/5TQ  <  0 , 

dMQ/dTQ  >0,  and  dMQ/dR  > 0. 

P 

•  For  equation  (3),  <?TQ/dML<0,  dTQ/dTL  <0 ,  and  <?TQ/<?R>0. 


(2) 

(3) 


TABLE  1:  Factors  Influencing  OJT  Capacity 


•  Resource  availability 

-  Availability  of  needed  quantity  and  quality  of 
training  facilities  and  equipment. 

-  Presence  of  professional  training  personnel. 

Presence  of  qualified  instructors. 

Availability  of  instructor  time  in  excess  of  that 
required  for  non-training  responsibilities, 
including  the  primary  mission,  record  keeping, 
and  administration. 

-  Influence  of  work  shift  scheduling  and  shop  manning 
on  the  presence  of  skilled  personnel  needed  for 
proficiency  training  within  a  work  center. 

-  Budgetary  constraints  on  participation  in  field 
training  detachment  (FTD)  programs. 

•  Job  characteristics 

Difficulty  of  tasks  to  be  trained. 

-  Frequency  of  occurrence  of  casks  requiring 
proficiency  training. 

•  Organizational  considerations 

Command  emphasis  on  the  importance  of  training. 
Morale. 

Trainee  characteristics,  such  as  ability, 
knowledge,  and  motivation. 

•  External  influences 

Seasonality. 

-  Weather  conditions,  such  as  temperature  and  snow. 


Finally,  applying  standards  —  determined  by  Air  Force  policy  —  speci¬ 
fying  the  minimum  acceptable  values  of  the  level  of  mission  performance 
(MiG) ,  the  quality  of  mission  performance  (MQ^) ,  and  the  quality  of 
training  (TQ^)  ,  the  OJT  capacity  of  the  unit  can  be  calculated  as  the 
maximum  training  load  supportable  by  the  unit.  This  calculation  is 
depicted  graphically  in  Figure  1.  Mathematically,  it  can  be  expressed 
as 


Maximize  TL 
subject  to  ML>ML® 
MQ>MQ0 
and  TQ  >TQ0 


Similarly,  if  Air  Force  policy-makers  can  develop  a  functional 
relationship  indicating  the  relative  desirability  of  different  com¬ 
binations  of  mission  performance  level,  mission  performance  quality, 
training  load,  and  training  quality,  the  desirable  OJT  capacity  of 
the  unit  can  also  be  estimated.  This  estimation  can  be  denoted  math¬ 
ematically  as 


Maximize  D 
TL 

subject  to 

and 

where 


(ML,  MQ,  TL,  TQ) 

ML>ML° 

MQ>MQ° 

TQ>TQ0, 

D  (ML,  MQ,  TL,  TQ)  is  the  unit's  "desirability" 

function. 


Since,  other  things  being  equal,  the  unit  prefers  higher  levels  of  each 
of  the  "desirability"  function's  independent  variables  to  lower  levels 
of  the  same  variable,  each  of  the  function's  first-order  partial  deriv¬ 
atives  can  reasonably  be  expected  to  be  positive. 

The  solution  of  this  problem  will  indicate  the  optimal  mission 
performance  level,  mission  performance  quality,  training  load,  and 
training  quality  that  can  be  achieved  by  the  unit  —  relative  to  its 
own  criterion  of  desirability  —  while  maintaining  acceptable  values 
of  mission  performance  level,  mission  performance  quality,  and  training 
quality.  It  is  conceivable  that,  in  many  situations,  the  minimum 
acceptability  conditions  will  not  impose  binding  constraints  on  the 
solution.  Nevertheless,  they  must  be  included  in  the  formulation  of 
the  problem  to  preclude  any  unacceptable  degradations  of  the  corre¬ 
sponding  variables  of  primary  interest. 
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4.0  FRAMEWORK  FOR  MODEL  DEMONSTRATION 


To  provide  a  basis  for  empirically  testing  the  conceptual  model 
of  GJT  capacity  formulated  in  the  preceding  section,  two  preparatory 
tasks  have  been  performed: 

e  Selection  of  the  specific  Air  Force  operational  units 
for  which  the  application  of  the  model  has  been  demon¬ 
strated. 

•  Identification  of  adequate  sources  of  the  various  types 
of  data  required  for  application  of  the  mode!  for  each 
selected  unit. 

The  outcomes  of  these  two  tasks  are  discussed  in  the  following  two 
subsections. 


4.1  Selection  of  Operational  Units 


To  assure  a  reasonably  thorough  assessment  of  the  potential  range 
of  application  of  the  OJT  capacity  estimation  methodology,  empirical 
tests  of  the  conceptual  model  of  OJT  capacity  have  been  conducted  for 
two  different  broad  classes  of  Air  Force  operational  units: 

•  Units  performing  maintenance  on  aircraft  assigned  to 
primary  flying  missions. 

•  Units  performing  no  aircraft  maintenance  activities. 

This  grouping  has  been  established  in  recognition  of  the  clear  distinc¬ 
tion  between  the  direct  association  of  aircraft  maintenance  activities 
with  the  level  and  quality  of  primary  mission  performance,  and  the  much 
more  indirect  relationships  of  primary  mission  performance  with  most 
other  types  of  activity.  This  distinction  has  clear  practical  signifi¬ 
cance  for  this  study.  Specifically,  in  aircraft  maintenance  units,  OJT 
efforts  can  reasonably  be  expected  to  be  directly  relatable  to  measures 
of  primary  mission  performance.  Conversely,  for  other  units,  OJT  activ¬ 
ities  are  likely,  at  best,  to  be  relatable  to  instrumental  indicators 
of  unit  performance  that  are  believed  to  have  some  indirect  ultimate 
influence  on  primary  mission  performance.  Consequently,  to  investigate 
the  potential  range  of  application  of  the  OJT  capacity  estimation  meth¬ 
odology,  empirical  tests  of  the  methodology  must  be  conducted  for  both 
of  these  broad  classes  of  Air  Force  operational  units. 


21 


FHEGBOlin  HIM 


•HOT  FILMS) 


A  total  of  six  operational  units  have  been  examined  in  this  study. 
Four  are  aircraft  maintenance  units,  three  attached  to  wings  equipped 
with  F-4  aircraft  and  one  attached  to  a  wing  with  RF-4  aircraft. *2  The 
fifth  unit  is  a  transportation  squadron,  responsible  primarily  for 
vehicle  maintenance.  The  sixth  is  a  supply  squadron. 


4. 2  Data  Sources 


To  perform  a  comprehensive  empirical  demonstration  of  the  OJT 
capacity  estimation  methodology,  six  categories  of  data  must  be  obtained 
These  categories  consist  of  indicators  of 

•  The  level  of  mission  performance. 

•  The  quality  of  mission  performance. 

•  The  amount  of  training  provided. 

•  The  quality  of  training. 

•  The  availability  of  resources. 

•  External  and  other  factors. 

For  the  current  study,  no  time  or  resources  have  been  allocated  for 
primary  data  collection  efforts.  Consequently,  all  data  used  in  the 
empirical  analyses  have  been  obtained  from  existing  secondary  sources. 

Ideally,  the  data  should  describe  activities  and  resources  at  the 
level  of  individual  work  centers.  It  is  at  this  level  that  the  inter¬ 
actions  among  resources,  training  loads,  and  mission  performance  that, 
in  combination,  determine  OJT  capacity  can  reasonably  be  expected  to 
occur  most  strongly  and  directly. 

Unfortunately,  none  of  the  operational  units  examined  In  the  study 
routinely  develop  and  maintain  the  desired  types  of  data  at  the  work 
center  level.  Although  it  is  relatively  simple  to  obtain  suitable 
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Part  of  the  reason  for  examining  these  four  units  has  been  to 
compare  the  relationships  between  training  loads  and  mission  perform¬ 
ance  for  two  different  types  of  aircraft  maintenance  organization: 
Production  Oriented  Maintenance  Organization  (POMO)  and  traditional 
organization.  The  results  of  this  comparison  are  reported  in  P.  J. 
DeLeo  and  T.  R.  Bell,  Estimating  the  Capacity  of  Air  Force  Maintenance 
Organizations  to  Conduct  On-the-Job  Training,  Lowry  AFB,  Colorado, 

Air  Force  Human  Resources  Laboratory,  Technical  Training  Division, 
December  1979. 
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information  about  the  current  characteristics  of  any  work  center,  it 
is  virtually  impossible  to  obtain  this  information  for  enough  histor¬ 
ical  time  periods  to  permit  empirical  demonstration  of  the  model. 


For  example,  in  the  aircraft  maintenance  units,  the  Maintenance 
Management  Information  Control  System  (MMICS)  appears  to  be  an  ideal 
source  of  information.  Among  its  many  capabilities,  MMICS  contains 
data  concerning  "...required  training  courses,  inspection  and  special 
qualifications,  OJT  training  information,  training  statuses,  due  and 
completed  dates,  and  an  individual's  complete  training  history. 
Moreover,  this  information  is  recorded  for  virtually  all  training 
activities  pertinent  to  OJT,  including  "...proficiency,  qualification, 
certification,  resident/FTD ,  dual  channel  OJT  and  locally  established 
requirements."^  However,  MMICS  does  not  systematically  retain  data 
describing  a  unit's  historical  training  activities.  Whenever  airmen 
are  transferred  to  other  units,  all  of  their  records  are  deleted  from 
the  data  stored  in  MMICS  for  the  units  from  which  they  are  departing. 
Moreover,  printed  copies  of  MMICS  data  are  retained  as  a  safeguard 
against  inadvertent  information  losses  for  at  most  3  months.  Conse¬ 
quently,  MMICS  cannot  provide  a  sufficiently  long  sequence  of  histo¬ 
rical  training  and  mission  performance  information  to  permit  its  use 
as  a  data  source  in  this  study. 

As  a  direct  result  of  such  data  limitations,  it  has  been  impos¬ 
sible  to  perform  any  empirical  demonstrations  of  the  OJT  capacity 
estimation  methodology  at  the  work  center  level.  Rather,  the  method¬ 
ology  has  been  applied  at  the  squadron  level  for  the  transportation 
and  supply  units,  and  at  the  wing  level  for  the  aircraft  maintenance 
units.  Only  at  these  higher  organizational  levels  could  a  sufficient 
amount  of  existing  historical  data  be  obtained  to  permit  empirical 
analysis. 

For  the  aircraft  maintenance  units,  the  data  used  to  demonstrate 
the  OJT  capacity  estimation  methodology  have  been  assembled  from  six 
different  sources.  Most  of  these  information  sources  are  printed 
summaries  published  by  various  wing- level  analysis  and  evaluation 
offices. 


Monthly  Maintenance  Summary,  Part  I,  prepared  by  the 
wing  maintenance  analysis  office,  provides  numerous 
monthly  Indicators  of  wing  mission  performance. 
Historical  records  of  the  Indicators  are  reported  in 
each  summary  for  as  many  as  12  consecutive  months. 
Moreover,  because  this  information  is  submitted  to  the 
analysis  staff  at  MAJCOM  (Major  Command)  headquarters, 
all  wings  employ  the  same  format  in  this  document. 


^Department  of  the  Air  Force 
Control  System  (MMICS) ,  Volune  I, 
November  1977,  p.  10-1. 


,  Maintenance  Management  Information 
AFM  66-278  (C12) ,  Washington,  D.C. , 


•  Monthly  Maintenance  Summary,  Part  IV, ^  also  compiled 
by  the  maintenance  analysis  office,  contains  a  summary 
of  quality  control  evaluations.  Two  types  of  evalua¬ 
tions  are  reported:  personnel  evaluations  appraising 
the  adequacy  of  task  performance  as  determined  by 
directly  observing  qualified  airmen  performing  the 
task,  and  task  evaluations  assessing  the  completeness 
and  correctness  of  task  performance  after  its  execu¬ 
tion.  To  the  extent  that  maintenance  task  performance 
contributes  to  primary  mission  performance,  these 
evaluations  can  be  used  as  indicators  of  the  quality 
of  mission  performance.  Alternatively,  to  the  extent 
that  successful  on-the-job  task  performance  indicates 
adequacy  of  training,  the  evaluations  can  be  inter¬ 
preted  as  measures  of  training  quality.  In  either 
event,  this  information  represents  potentially 
pertinent  data  for  the  study. 

•  Records  maintained  by  the  training  management  office 
specify  the  amount  of  formally  monitored  training 
conducted  in  the  aircraft  maintenance  units.  In  par¬ 
ticular,  these  records  report  UGT  enrollments  and 
completions.  However,  the  training  management  offices 
of  different  bases  employ  different  procedures  concern¬ 
ing  the  retention  of  historical  information. 

•  The  Daedalian  Monthly  Maintenance  Award  Report,  which 
also  publishes  monthly  UGT  enrollment,  can,  fortunately, 
normally  be  used  to  supplement  the  other  training 
records  to  obtain  an  uninterrupted  historical  sequence 
for  UGT  enrollment.  This  report  also  contains  infor¬ 
mation  about  a  variety  of  other  aircraft  maintenance 
activities  and,  hence,  can  often  provide  values  found 

to  be  missing  in  other  data  sources. 

e  The  Organizational  Manning  Report,  prepared  by  the 

programs  and  mobility  office,  provides  an  exact  record 
of  actual  personnel  assignments  in  individual  work 
centers.  This  report  identifies  the  work  center, 
duty  assignment,  and  Primary  Air  Force  Specialty  Code 
(PAFSC)  for  each  airman  assigned  to  the  unit;  and  it 
indicates  the  authorized  manning  levels  for  each  work 
center. 


14 

Monthly  Maintenance  Summary,  Parts  II  and  III,  contain  more 
detailed  information  about  the  maintenance  of  individual  aircraft  and 
specific  component  systems.  Thus,  these  reports  directly  reflect  the 
performance  of  individual  work  centers.  However,  as  explained  previ¬ 
ously,  the  absence  of  comparable  data  about  training  activities  pre¬ 
cludes  the  use  of  this  information. 


i 
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•  Records  maintained  by  the  Wing  Historian  can  occasion¬ 
ally  be  used  to  provide  information  missing  from  other 
data  sources.  However,  most  of  the  information  com¬ 
piled  by  this  office  pertains  to  the  entire  wing,  but 
does  not  describe  individual  squadrons  or  work  centers. 

Obviously,  many  of  these  data  sources  just  described  contain  infor¬ 
mation  that  overlaps  data  in  other  sources.  A  complete  compilation  of 
the  potentially  pertinent  indicators  reported  in  each  cited  data  source 
is  presented  in  Table  2.  Nevertheless,  undesirable  missing  information 
remains  for  many  important  variables  and  reduces  the  usable  size  of  the 
final  data  base.  For  some  indicators,  50  consecutive  monthly  observa¬ 
tions  are  available;  while  for  other  variables,  only  four  observations 
have  been  obtained. 

For  the  transportation  and  supply  squadrons,  fewer  sources  of 
pertinent  information  have  been  found.  Nevertheless,  the  identified 
sources  have  provided  uninterrupted  historical  data  series  for  all 
variables  of  interest  spanning  24  consecutive  months. 

•  For  the  transportation  squadron,  the  VIMS  (Vehicle 
Integrated  Managanent  System)  Monthly  Analysis, 
prepared  by  the  maintenance  analysis  branch  of  the 
squadron,  furnishes  several  monthly  measures  of  the 
performance  of  the  squadron' s  vehicle  maintenance 
mission.  Additional  unpublished  vehicle  maintenance 
performance  indicators  are  also  prepared  by  and 
available  from  the  squadron's  maintenance  analysis 
branch. 

•  For  the  supply  squadron,  internal  reports  compiled 

by  its  analysis  office  provide  many  monthly  indicators 
of  the  level  and  quality  of  the  squadron’s  logistic 
efforts.  For  each  of  the  squadrons,  historical  values 
of  the  indicators  are  presented  in  each  report  for  as 
many  as  12  consecutive  months. 

•  Records  of  the  amount  of  formally  monitored  training 
conducted  in  the  two  squadrons  are  maintained  by  the 
OJT  office  of  the  Consolidated  Base  Personnel  Office. 

These  records  indicate,  for  each  squadron,  the  total 
monthly  training  loads  and  upgrades  for  each  skill 
level,  and  the  overall  amounts  of  UGT  and  retraining 
conducted  in  each  month. 

•  Additional  records  compiled  by  the  OJT  office  specify 
the  actual  number  of  personnel  assigned  to  each 
squadron  in  each  month.  The  number  of  personnel 
authorized  for  each  squadron  throughout  each  quarter 
is  also  available  through  the  Consolidated  Base 
Personnel  Office. 


TABLE  2:  Aircraft  Maintenance  Unit  Data  Sources 


Data  Source 
(Document 
and/or  Office) 

Data  Item  's. 

Monthly 

Maintenance 

Summary,  Part  I 

Monthly 

Maintenance 

Summary,  Part  IV 

Daedalian 

Maintenance 

Award  Report 

Wing 

Historian 

Training 

Management 

Organizational  Manning 
Report  -  Programs 
and  Mobility 

Hama  Flown  Hours 

X 

X 

X 

Home  Sorties  Scheduled 

X 

X 

Home  Sorties  Flown 

X 

X 

X 

Rasa  Sorties  Flown  as  Scheduled 

X 

Cround  Aborts 

X 

X 

Air  Aborts 

X 

X 

Maintenance  Cancellations 

X 

Maintenance  Attrition  Rate 

X 

Maintenance  Late  Take-Offs 

X 

Broke  Aircraft 

X 

Aircraft  Recovered  Within  3  Hours 

X 

Aircraft  Recovered  Wichin  6  Hours 

X 

Maintenance  Scheduling  Effectiveness 

X 

X 

Possessed  Hours 

X 

MQf(S)a  Hours 

X 

mO((U)b  Hours 

X 

PMCMc  Hours 

X 

Percent  NMCM^ 

X 

X 

Percent  PMCMC 

X 

X 

QVIs«  Satisfactory  and  Unsatisfactory 

X 

X 

PE«f  Passed  and  Failed 

X 

Detected  Safety  Violations 

X 

Discrepancies  Count 

Repeats 

Recurs 

Cannot  Duplicates 

Base  Repair  Rate 

Mission  Capability 

X 

OJT  Upgrade  Trainees:  5  and  7  Level 

X 

X 

Manhours  Assigned 

Overtime  Manhours 

Training  Manhours 

Assignments  by  Duty  Skill  Level 

X 

X 

Authorizations  by  Duty  Skill  Level 

X 

X 

Aaslgments  Whose  Duty  AFS&  Differs 

Pram  Their  Primary  AFS® 

X 

FID  Enrollments 

X 

Maher  of  Ancillary  Training  Evants 

X 

Number  of  Assignments  Having 

PAPSC"  75 XXX 

X 

X 

N°i  al«* Ion  capable  due  to  scheduled  maintenance. 

"j|ot  alaalon  capable  due  to  unscheduled  maintenance, 
partially  mission  capable,  but  requiring  maintenance. 
Tot  mission  capable  due  to  maintenance. 

^Quality  verification  inspections. 

Personnel  evaluations. 

?Alr  Pores  specialty, 
primary  Air  Pores  specialty  coda. 
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•  Data  concerning  monthly  weather  conditions  have  been 
obtained  from  the  base  weather  station.  These  data 
describe  the  typical  base  weather  situation  in  terms 
of  monthly  maximum,  minimum,  and  average  temperature, 
snowfall,  and  precipitation  levels. 

Complete  compilations  of  the  potentially  pertinent  indicators 
available  from  each  cited  data  source  are  presented  for  the  transpor¬ 
tation  and  supply  squadrons  in  Tables  3  and  4,  respectively. 


TABLE  3:  Transportation  Squadron  Data  Sources 


n.  Data  Sources 

N.  (Document 

n.  and/or  Office) 

Data  Item  \ 

VIMS  Monthly  Analysis 

Maintenance  Analysis 
Branch,  Transpor¬ 
tation  Squadron 

OJT  Office, 
Consolidated  Base 
Personnel  Office 

Consolidated  Base 

Personnel  Office 

L  . 

Base  Weather  Station 

VDMa  hours 

X 

Estimated  direct  labor  hours  required 

X 

Actual  direct  labor  hours  applied 

X 

UGT  enrollment 

X 

OJT  upgrades  to  3,  5,  and  7  levels 

X 

Personnel  assigned 

X 

Personnel  authorized 

X 

Assigned  personnel  fully  trained 

for  grade 

X 

Average  monthly  temperature 

X 

Average  monthly  snowfall 

X 

Average  monthly  precipitation 

X 

vehicle  down  for  maintenance 


TABLE  4:  Supply  Squadron  Data  Sources 


Data  Sources 
(Document 
and/or  Office) 

Data  Item 

Analysis  Office, 
Supply  Squadron 

OJT  Office, 
Consolidated  Base 
Personnel  Office 

Consolidated  Base 
Personnel  Office 

Base  Weather  Station 

Reverse  post  rate 

X 

Inventory  adjustments 

X 

RCAa  units  over  and  short 

X 

Backorders 

X 

EOQb  units  over  and  short 

X 

Total  auditable  documents 

X 

Line  items  issued 

X 

Line  items  issued  and  backordered 

X 

RCAa  units 

X 

Total  transactions 

X 

UGT  enrollment 

X 

OJT  upgrades  to  3,  5,  and  7  levels 

X 

Personnel  assigned 

X 

Personnel  authorized 

X 

Assigned  personnel  fully  trained  for  grade 

X 

Average  monthly  temperature 

X 

Average  monthly  snowfall 

X 

Average  monthly  precipitation 

X 

aRepair  cycle  assets. 
^Economic  order  quantity 


5.0  EMPIRICAL  RESULTS 


i; 


Despite  the  limited  availability  of  suitable  data  from  existing 
sources,  a  persistent  and  strong  relationship  between  mission  perform¬ 
ance  and  training  load  has  been  estimated  for  all  operational  units 
studied.  The  derivation  of  statistically  significant  relationships 
in  spite  of  the  relatively  poor  quality  of  the  obtainable  indicators 
and  the  absence  of  some  important  explanatory  variables  is  most  encour¬ 
aging.  This  favorable  result  suggests  that  further  study  can  reason¬ 
ably  be  expected  to  produce  very  useful  methodologies  for  estimating 
OJT  capacity. 

The  data  limitations  encountered  in  the  study  have  caused  four 
basic  problems  in  empirically  demonstrating  the  application  of  the 
methodology: 

•  The  available  indicators  of  mission  performance  seldom 
are  unambiguous  measures  of  either  the  level  or  the 
quality  of  performance.  Consequently,  the  empirical 
results  derived  for  these  indicators  are  often  amenable 
to  varying  and  Inconclusive  interpretations. 

•  Satisfactory  indicators  of  the  quality  of  training  are 
generally  unavailable.  Hence,  no  relationships  respon¬ 
sive  to  the  quality  of  training  have  been  derived. 

•  The  existing  indicators  of  training  load  essentially 
measure  only  gross  UGT  activity.  No  measures  of  other 
types  of  training,  or  of  trainees'  progress  within  UGT, 
are  available. 

•  Indicators  of  other  resources  and  external  effects  are 
restricted  to  measures  of  enlisted  personnel  strength 
and  weather  conditions. 

If  more  directly  applicable  data  were  made  available  —  by  retaining 
data  now  routinely  discarded  or  by  systematically  collecting  new  data 
—  these  difficulties  would  be  sharply  reduced. 

Nevertheless,  the  empirical  results  collectively  demonstrate  a 
convincing  relationship  between  the  mission  performance  and  training 
loads  of  Air  Force  operational  units.  These  results,  developed  by 
applying  multiple  linear  regression  analysis  to  the  conceptual  model 
of  OJT  capacity  formulated  in  Section  3.0,  are  presented  in  the 
remainder  of  this  section. 

The  results  for  the  three  different  types  of  operational  units 
appear  in  the  order  of  increasing  complexity.  This  order  of  presenta¬ 
tion  will  permit  the  reader  to  comprehend  the  less  complex  results 
fully  before  being  confronted  with  the  results  that  are  more  difficult 

to  interpret. 
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Thus,  the  results  derived  for  the  transportation  squadron  appear 
in  Section  5.1;  the  results  for  the  supply  squadron  are  reported  in  the 
following  section;  and  the  results  for  the  aircraft  maintenance  units 
are  summarized  in  Section  5.3.  The  transportation  squadron  offers  only 
one  good  measure  of  mission  performance,  and  the  available  data  for  the 
explanatory  variables  are  limited  also.  The  supply  squadron  provides 
two  indicators  of  mission  performance.  However,  these  measures  are  not 
very  well  defined  and,  hence,  present  problems  in  interpretation  of  the 
empirical  results.  Finally,  the  aircraft  maintenance  units  furnish  the 
largest  amounts  of  data,  but  also  entail  the  greatest  difficulties  in 
interpretation. 


5.1  Transportation  Squadron 


Due  to  data  limitations,  the  only  equation  from  the  conceptual 
model  of  OJT  capacity  that  has  been  estimated  for  the  transportation 
squadron  is  the  equation  describing  the  quality  of  mission  performance 

MQ  -  g  (ML,  TL,  TQ,  TQp,  R,  E) 
where,  as  before. 


MQ  =  the  quality  of  mission  performance. 

ML  *  the  level  of  mission  performance. 

TL  *  the  amount  of  training  provided. 

TQ  -  the  quality  of  current  training. 

TQp  *  the  quality  of  previous  training. 

R  *  the  resources  available  to  the  squadron. 

E  *  external,  and  other,  factors. 

In  addition,  because  no  suitable  data  have  been  found  measuring  either 
the  level  of  mission  performance  or  the  quality  of  training  achieved  in 
the  squadron,  the  relationship  actually  examined  has  been 


MQ  -  G  (TL,  R,  E) 


Four  different  formulations  of  this  basic  relationship  in  which  mission 
performance  quality  is  associated  with  training  load,  resource  avail¬ 
ability,  and  external  factors  have  been  estimated  for  the  transportation 

squadron. 
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Only  one  acceptable  indicator  of  the  dependent  variable,  the 
quality  of  mission  performance,  has  been  uncovered  for  this  squadron. 

This  indicator  is  the  total  number  of  hours  that  vehicles  have  been 
down  for  maintenance  by  the  transportation  squadron  during  each  month 
(VDMHRS).  It  represents  a  conceptually  satisfactory  measure  of  the 
quality  of  mission  performance  since,  other  things  being  equal,  the 
greater  are  the  number  of  hours  that  vehicles  have  been  done  for 
maintenance  by  the  squadron,  the  less  productive  the  squadron's  main¬ 
tenance  efforts  must  have  been.  Unfortunately,  other  things  are  rarely 
equal.  The  number  of  miles  that  vehicles  are  driven,  the  rate  at  which 
vehicles  arrive  for  maintenance,  and  the  difficulty  of  the  maintenance 
required  by  the  newly  arrived  vehicles  all  vary  from  month  to  month. 
Consequently,  VDMHRS  unavoidably  reflects  these,  and  other,  aspects 
of  changes  in  the  workload  confronting  the  squadron.  Moreover,  since 
no  data  have  been  found  describing  any  such  external  factors,  it  is 
impossible  to  adjust  for  these  effects  empirically.  Rather,  the 
possible  influence  of  any  excluded  explanatory  variables  should  be 
taken  into  consideration  when  interpreting  the  empirical  results. 

The  explanatory  variables  entered  into  the  estimated  relationships 
include  indicators  of  the  amount  of  training  provided  (TL) ,  squadron 
resources  (R) ,  and  some  external  factors  (E) . 

The  basic  variable  used  as  an  indicator  of  the  amount  of  training 
provided  in  the  squadron  is  number  of  enlisted  personnel  enrolled  in 
UGT  in  each  month  (TLOAD) .  This  variable  captures  most  of  the  squadron's 
training  load.  However,  it  does  not  reflect  informal  OJT  activities 
such  as  qualification  training,  nor  does  it  account  for  any  progress 
achieved  by  trainees  as  their  OJT  proceeds.  In  two  of  the  relationships 
estimated  for  this  squadron,  TLOAD  is  entered  directly  as  an  explanatory 
variable.  In  the  remaining  two  formulations,  TLOAD  has  been  divided  by 
the  number  of  enlisted  personnel  assigned  to  the  squadron  to  produce  a 
new  explanatory  variable,  the  proportion  of  the  squadron's  enlisted 
personnel  enrolled  in  UGT  in  each  month  (RELTLD) . 

Only  one  basic  variable  has  been  found  to  measure  the  resources 
available  to  the  squadron.  This  variable  specifies  the  number  of 
enlisted  personnel  assigned  to  the  squadron  in  each  month.  As  indi¬ 
cated  previously,  it  has  been  used  as  the  denominator  of  the  explana¬ 
tory  variable  RELTLD  in  two  of  the  estimated  relationships.  Moreover, 
in  one  other  formulation,  it  has  been  divided  by  the  number  of  enlisted 
personnel  authorized  for  the  squadron  in  tsch  month  to  create  an  addi¬ 
tional  explanatory  variable,  the  proportion  of  the  squadron’s  author¬ 
ized  enlisted  strength  actually  assigned  to  the  squadron  in  each  month 
(RELSTR) . 


Finally,  two  somewhat  crude  measures  of  monthly  weather  conditions 
have  been  included  as  indicators  of  external  factors.  These  variables 
are  the  average  snowfall  in  each  month  (SNOW)  and  the  deviation  of  the 
average  temperature  in  each  month  from  the  average  annual  temperature 
at  the  Air  Force  base  housing  the  squadron  (DEVTMP). 

Thus,  assuming  a  basic  linear  relationship,  the  general  form  of 
the  equation  actually  estimated  for  the  transportation  squadron  is 

MQ  -  a  +  bTL  +  cR  +  dE 

where  MQ  -  VDMHRS. 

TL  -  TLOAD  or  RELTLD. 

R  -  RELSTR. 

E  -  SNOW  and/or  DEVTMP. 

Table  5  presents  the  empirical  results  obtained  by  applying  mul¬ 
tiple  linear  regression  analysis  to  derive  estimates  of  four  different 
formulations  of  this  general  equation.  These  represent  all  versions 
of  the  general  equation  for  which  all  estimated  coefficients  are  statis¬ 
tically  significant  at  the  5  percent  level.  All  attempts  to  establish 
versions  of  the  equation  containing  the  constant  term,  a,  have  produced 
statistically  insignificant  estimates  of  the  constant.  Consequently, 
none  of  the  formulations  reported  in  Table  5  contains  a  constant  term. 

Despite  all  of  the  inadequacies  in  the  data,  these  results  uni¬ 
formly  reveal  a  statistically  significant  positive  relationship  between 
the  level  of  training  provided  in  the  transportation  squadron  and 
VDMHRS.  Although  the  overall  quality  of  the  estimated  relationships, 
as  measured  by  R^,  is  not  exceptional,  the  generally  high  values  of 
the  t-statisticsl5  associated  with  the  training  load  variables  clearly 
and  strongly  indicate  that  increases  in  the  amount  of  training  con¬ 
ducted  in  the  transportation  squadron  are  significantly  correlated  with 
reductions  in  the  quality  of  the  squadron's  mission  performance.  In 
addition,  the  results  suggest  that  RELTLD,  the  unit's  training  load 
relative  to  its  assigned  enlisted  strength,  provides  a  better  descrip¬ 
tion  of  this  relationship  than  does  TLOAD,  the  absolute  amount  of 
training  conducted  in  the  squadron. 


Each  t-statistic  reported  in  Table  5  indicates  the  number  of 
standard  deviations  by  which  the  corresponding  estimated  regression 
coefficient  differs  from  zero.  The  significance  level  associated  with 
the  t-statistic,  therefore,  specifies  the  probability  that  the  esti¬ 
mated  coefficient  would  have  been  derived  empirically  if  the  true 
value  of  the  coefficient  were  zero. 
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TABLE  5:  Bapirical  Results  of  Transportation  Squadron  Regressions 


5.2  Supply  Squadron 


As  in  the  case  of  the  transportation  squadron,  the  lack  of  suit¬ 
able  data  for  the  supply  squadron  permits  the  estimation  of  only  one 
equation  from  the  conceptual  model  of  OJT  capacity  formulated  in 
Section  3.0.  Moreover,  as  before,  the  only  relationship  that  can  be 
empirically  investigated  is  the  equation  describing  the  quality  of 
mission  performance 

MQ  “  g  (ML,  TL,  TQ,  TQp,  R,  E). 

However,  for  the  supply  squadron,  no  data  have  been  located  describing 
either  the  level  of  training  quality  achieved  or  any  external  factors 
likely  to  affect  supply  operations.  Consequently,  the  relationship 
actually  analyzed  for  this  squadron  has  been 

MQ  -  G  (ML,  TL,  R). 

Fifteen  different  formulations  of  this  basic  relationship,  in  which 
mission  performance  quality  is  associated  with  mission  performance 
level,  training  load,  and  resource  availability,  have  been  estimated 
for  the  supply  squadron. 

Since  no  single  definitive  measure  of  the  dependent  variable,  the 
quality  of  mission  performance,  has  been  uncovered,  two  different  mea¬ 
sures  considered  to  be  reasonable  indicators  of  poor  mission  perform¬ 
ance  quality  have  been  employed  as  dependent  variables.  The  first 
variable  measures  adjustments  in  recorded  inventory  balances  (INVADJ) ; 
while  the  second  measures  reverse  postings  (RPOST)  —  computer  entries 
that  reverse  previous  erroneous  entries.  Both  variables  measure, 
basically,  personnel  errors.  Hence,  both  should  capture  some  aspects 
of  the  quality  of  mission  performance.  Yet,  they  reflect  neither  the 
activities  of  all  personnel  in  the  squadron,  nor  all  facets  of  the 
overall  performance  of  the  unit. 

Three  fundamental  types  of  explanatory  variables  enter  into  the 
estimated  relationships,  including  indicators  of  the  level  of  mission 
performance  (ML),  the  amount  of  training  provided  (TL) ,  and  squadron 
resources  (R). 

Three  different  variables  have  been  tested  as  indicators  of  the 
level  of  mission  performance: 

•  ISSUED:  the  number  of  items  Issued  by  the  squadron 
in  each  month. 

•  TRANS:  the  nunber  of  computer  transactions  processed 
by  the  squadron  in  each  month. 


DCC:  the  number  of  auditable  documents  generated  by 
the  squadron  in  each  month. 


None  of  these  variables  represents  an  ideal  measure  of  the  level  of 
mission  performance,  but  each  does  capture  some  aspect  of  the  amount 
of  supply  services  provided. 


Once  again,  the  basic  variable  used  to  indicate  the  amount  of 
training  conducted  in  the  squadron  is  the  number  of  enlisted  personnel 
enrolled  In  UGT  in  each  month  (TLOAD).  In  seven  of  the  relationships 
estimated  for  this  unit,  TLOAD  is  entered  directly  as  an  explanatory 
variable.  In  seven  others,  TLOAD  has  been  divided  by  the  number  of 
enlisted  personnel  assigned  to  the  squadron  to  create  a  new  explana¬ 
tory  variable,  the  proportion  of  the  enlisted  personnel  enrolled  in 
UGT  each  month  (RELTLD).  Finally,  to  examine  the  effect  on  mission 
performance  quality  of  changes  in  the  amount  of  training  conducted 
over  time,  one  relationship  has  been  estimated  using  the  difference 
between  the  squadron’s  training  loads  in  consecutive  months  (DELTLD) 
as  an  explanatory  variable. 

Only  one  variable  has  been  discovered  to  measure  the  resources 
available  to  the  squadron.  This  variable  specifies  the  number  of 
enlisted  personnel  assigned  to  the  unit  and,  as  indicated  previously, 
has  been  used  as  the  denominator  of  the  variable  RELTLD  in  seven  of 
the  estimated  relationships. 


Therefore,  assuming  again  a  linear  relationship,  the  general  form 
of  the  equation  actually  estimated  for  the  supply  squadron  is 
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a  +  b  ML  +  c  TL 

MQ 

- 

INVADJ  or  RPOST. 

ML 

m 

ISSUED,  TRANS,  or 

DCC. 

TL 

m 

TLOAD,  RELTLD,  or 

DELTLD. 

Tables  6  and  7  present  the  empirical  results  obtained  by  applying 
multiple  linear  regression  analysis  to  generate  estimates  of  15  differ¬ 
ent  formulations  of  this  general  equation.  These  formulations  include 
all  versions  of  the  general  equation  for  which  all  estimated  coeffi¬ 
cients  are  statistically  significant  at  the  5  percent  level.  In 
addition,  to  facilitate  comparison  of  the  results  obtained  for  the 
two  alternative  indicators  of  the  quality  of  mission  performance,  the 
tables  present  the  estimates  derived  for  each  of  these  dependent  vari¬ 
ables  using  as  explanatory  variables  all  pairwise  combinations  of 
TLOAD  or  RELTLD  with  ISSUED,  TRANS,  or  DCC.  The  results  derived  using 
INVADJ  as  the  dependent  variable  appear  in  Table  6;  while  Table  7  dis¬ 
plays  the  results  obtained  using  RFOST  as  the  dependent  variable. 


TABLE  6:  Bnpirical  Results  of  Supply  Squadron 

Regressions  with  INVADJ  as  Dependent  Variable 


Empirical  Results  of  Supply  Squadron 
Regressions  with  RPOST  as  Dependent  Variable 


'Significantly  different  from  0  at  the  10  percent  level. 


The  regression  coefficients  and  t-statistics  reported  in  Table  6 
Indicate  that  INVADJ  is  uniformly  positively  related  to  the  amount  of 
training  provided  in  the  unit,  although  the  overall  quality  of  the 
estimated  relationships,  as  measured  by  R^,  is  not  exceptional.  These 
empirical  results  also  suggest  that  the  incidence  of  personnel  errors 
leading  to  inventory  adjustments  is  not  influenced  significantly  by 
the  volume  of  work  processed  by  the  squadron. 

Conversely,  as  revealed  in  Table  7,  RPOST  is  consistently  nega¬ 
tively  related  to  all  indicators  of  the  level  of  training  conducted 
by  the  squadron.  Although  this  outcome  is  contrary  to  prior  expecta¬ 
tions,  two  possible  explanations  for  this  empirical  result  appear 
reasonable:  either  trainees  perform  relatively  menial  tasks,  such 
as  entering  information  into  a  computer,  more  carefully  than  do  expe¬ 
rienced  personnel,  or  else  trainees  may  be  more  closely  supervised 
than  are  the  fully  trained  airmen.  The  results  presented  in  Table  7 
also  indicate  that  the  incidence  of  personnel  errors  resulting  in 
reverse  postings  is  significantly  positively  related  to  the  amount 
of  work  processed  by  the  squadron.  In  fact,  the  t-statistics  suggest 
that,  in  general,  RPOST  is  more  strongly  influenced  by  workload  con¬ 
siderations  than  by  training  load  effects. 


To  provide  some  perspective  to  these  apparently  contradictory 
sets  of  empirical  results,  observe  that,  in  contrast  to  RPOST,  INVADJ 
reflects  a  much  broader  range  of  personnel  errors,  including  errors 
associated  with  the  more  complex,  less  menial  supply  operations. 

Thus,  INVADJ  is  a  more  comprehensive  indicator  of  the  quality  of 
mission  performance  than  is  RPOST;  and  the  results  reported  in  Table  6 
represent  a  more  thorough  and  convincing  test  of  the  overall  relation¬ 
ship  between  mission  performance  quality  and  squadron  training  loads 
than  do  the  results  derived  using  RPOST  as  the  dependent  variable. 


5.3  Aircraft  Maintenance  Units 


The  four  aircraft  maintenance  units  have  presented  the  greatest 
opportunity  in  this  study  for  empirically  testing  all  components  of 
the  conceptual  model  of  OJT  capacity.  Despite  considerable  variabil¬ 
ity  in  data  availability  and  quality,  reasonable  indicators  are  avail¬ 
able  for  the  level  of  mission  performance,  the  quality  of  mission 
performance,  the  amount  of  training  conducted,  and  the  quality  of 
training  provided  in  each  of  the  units. 

Consequently,  all  three  basic  equations  in  the  conceptual  model 
of  OJT  capacity  have  been  examined  for  the  aircraft  maintenance  units 

ML  -  f  (TL,  TQ  ,  R,  E) 

P 


MQ  -  g  (ML,  TL,  TQ,  TQp,  R,  E) 
TQ  -  h  (ML,  TL,  R,  E) 


where,  once  again 


ML  *  the  level  of  mission  performance. 

MQ  ■  the  quality  of  mission  performance. 

TQ  ■  the  quality  of  :urrent  training. 

TQp  *  the  quality  of  previous  training. 

TL  *  the  amount  of  training  provided. 

R  *  the  resources  available  to  the  unit. 

E  *  external,  and  other,  factors. 


Only  one  usable  indicator  of  the  level  of  mission  performance  has 
been  uncovered  for  the  aircraft  maintenance  units.  This  indicator  is 
the  ratio  of  the  total  number  of  flying-hours  generated  by  the  aircraft 
maintained  by  a  unit  in  each  month,  to  the  total  number  of  person-days 
for  which  enlisted  personnel  have  been  assigned  to  the  unit  during  the 
month  (FLYHRS) . 

In  contrast,  17  different  indicators  of  the  quality  of  mission 
performance  have  been  developed.  To  simplify  interpretation  of  the 
empirical  results,  each  indicator  has  been  formulated  so  that  its 
value  increases  as  mission  performance  quality  improves.  The  indi¬ 
cators,  their  interpretations,  and  their  methods  of  calculation  are 

•  NGBORTS:  the  absence  of  flights  aborted  before  takeoff 

■  1.0  -  (the  number  of  ground  aborts  in  each 
month/the  number  of  sorties  flown  during  the 
month) . 

•  NABORTS:  the  absence  of  flights  aborted  after  takeoff 

*  1.0  -  (the  number  of  air  aborts  in  each 
month/the  number  of  sorties  flown  during  the 
month) . 


•  NBORTS:  the  absence  of  aborted  flights 

■  1.0  -  (the  total  number  of  air  and  ground 
aborts  in  each  month/ the  number  of  sorties 
flown  during  the  month). 

•  NMNTC:  the  absence  of  maintenance-related  sortie 

cancellations 

•  1.0  -  (the  number  of  sortie  cancellations 
related  to  maintenance  in  each  month/ the 
number  of  sorties  flown  during  the  month). 
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NMATTR: 


the  absence  of  maintenance  attrition 
«  1.0  -  maintenance  attrition  rate  in  each 
month. 

•  NFAIL:  the  absence  of  aircraft  and  component  failures 

■  1.0  -  (the  number  of  aircraft  and  component 
failures  in  each  month/the  number  of  sorties 
flown  during  the  month). 

•  RR3HR:  the  3-hour  recovery  rate 

=  the  number  of  aircraft  repaired  and  ready 
for  flight  within  3  hours  during  each  month/ 
the  number  of  aircraft  requiring  maintenance 
in  the  month. 

•  RR6HR:  the  6-hour  recovery  rate 

■  the  number  of  aircraft  repaired  and  ready 
for  flight  within  6  hours  during  each  month/ 
the  number  of  aircraft  requiring  maintenance 
in  the  month. 

•  MEFFC:  the  maintenance  scheduling  effectiveness  rate 

*  a  maintenance  task  accomplishment  index 
developed  by  the  Air  Force. 

•  ONTIME:  the  ability  to  take  off  on  schedule 

*  1.0  -  (the  number  of  late  takeoffs  in  each 
month/ the  number  of  sorties  flown  during  the 
month) . 

•  SORTYS:  the  ability  to  fulfill  the  programmed  flying 

schedule 

*  the  number  of  sorties  flown  as  scheduled 
during  each  month/ the  number  of  sorties 
scheduled  for  the  month. 

•  NNMCU:  the  proportion  of  aircraft  time  during  which 

available  aircraft  are  free  of  essential 
unscheduled  maintenance  requirements 

■  1.0  -  (the  total  number  of  hours  for  which 
aircraft  are  not  mission  capable  due  to  unsched¬ 
uled  maintenance  in  each  month/the  total  number 
of  hours  for  which  aircraft  are  available  to  the 
unit  during  the  month). 
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•  NMNCUS:  the  proportion  of  aircraft  time  during  which 

available  aircraft  are  free  of  essential  scheduled 
or  unscheduled  maintenance  requirements 

■  1.0  -  (the  total  number  of  hours  for  which  air¬ 
craft  are  not  mission  capable  due  to  scheduled  or 
unscheduled  maintenance  in  each  month/ the  total 
number  of  hours  for  which  aircraft  are  available 
to  the  unit  during  the  month). 

•  NPMOl:  the  proportion  of  aircraft  time  during  which 

available  aircraft  are  free  of  needed,  but  not 
essential,  maintenance  requirements 

■  1.0  -  (the  total  number  of  hours  for  which  air¬ 
craft  are  partially  mission  capable,  but  requiring 
maintenance,  in  each  month/ the  total  number  of 
hours  for  which  aircraft  are  available  to  the 
unit  during  the  month). 

•  NPCNM:  the  proportion  of  aircraft  time  during  which 

available  aircraft  are  completely  free  of 
maintenance  requirements 

■  1.0  -  (the  total  number  of  hours  for  which 
aircraft  are  either  not  mission  capable  due  to 
maintenance,  or  partially  mission  capable  but 
requiring  maintenance,  in  each  month/ the  total 
number  of  hours  for  which  aircraft  are  available 
to  the  unit  during  the  month). 

e  NRCWU:  the  absence  of  recurring  equipment  failures 

*=  1.0  -  (the  number  of  maintenance  writeups  that 
recur  within  the  subsequent  three  sorties,  but 
do  not  occur  on  consecutive  sorties,  in  each 
month/the  total  number  of  maintenance  writeups 
in  the  month) . 

•  BREPR:  the  base  repair  rate 

«  an  index  developed  by  the  Air  Force  describing 
the  proportion  of  maintenance  tasks  performed  on 
location  without  external  assistance  in  each 
month. 

Although  each  of  these  indicators  reflects  some  aspects  of  the 
quality  of  mission  performance  attributable  to  the  aircraft  maintenance 
units,  none  represents  a  comprehensive  and  conclusive  measure  of  mission 
performance  quality.  Consequently,  all  17  indicators  have  been  used  in 
Che  empirical  testing  of  the  conceptual  model  of  OJT  capacity  for  the 
aircraft  maintenance  units. 
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Similarly,  three  indicators  of  the  quality  of  training  provided  by 
the  units  have  been  included  in  the  analysis.  One  of  these  indicators 
assesses  the  quality  of  the  maintenance  work  performed  by  trained  per¬ 
sonnel;  and  two  appraise  the  quality  of  the  maintenance  practices 
employed  by  such  personnel  while  performing  the  work.  The  indicators, 
their  interpretations,  and  their  methods  of  calculation  are 

•  QINSPC:  the  satisfactory  quality  of  completed  maintenance 

tasks 

■  the  number  of  quality  verification  inspections 
in  which  completed  tasks  are  judged  satisfactory 
in  each  month/the  number  of  quality  verification 
inspections  conducted  during  the  month. 

•  PINSPC:  the  satisfactory  quality  of  maintenance  task 

performance 

-  the  number  of  personnel  inspections  in  which 
personnel  are  observed  performing  tasks  satis¬ 
factorily  in  each  month/the  number  of  personnel 
inspections  conducted  during  the  month. 

•  SAFET:  the  safe  performance  of  maintenance  tasks 

■  1.0  -  (the  number  of  safety  violations 
detected  during  personnel  inspections  in  each 
month/the  number  of  personnel  inspections  con¬ 
ducted  during  the  month). 

All  of  these  indicators  are  measures  of  on-the-job  performance  subse¬ 
quent  to  training.  Thus,  while  they  are  proffered  as  assessments  of 
training  quality,  they  can  be  interpreted  equally  justifiably  as 
indices  of  mission  performance  quality.  This  ambiguity  is  unavoidable, 
and  should  be  taken  into  consideration  when  interpreting  the  empirical 
results. 

The  amount  of  training  provided  in  the  aircraft  maintenance  units 
is  also  represented  by  three  indicators: 

•  PTRN:  the  amount  of  proficiency  training  conducted 

•  the  number  of  trainees  enrolled  in  UGT  to 
the  5-skill  level  in  each  month/the  number  of 
5-,  7-,  and  9-skill  level  personnel  assigned 
to  the  unit  during  the  month. 

•  STRN:  the  amount  of  supervision  training  conducted 

■  the  number  of  trainees  enrolled  in  UGT  to 
the  7-skill  level  in  each  month/the  number  of 
7-  and  9-skill  level  personnel  assigned  to  the 
unit  during  the  month. 
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•  RE57:  the  amount  of  formal  training  conducted 

■  the  number  of  trainees  enrolled  in  UGT  to 
the  5-skill  level  and  to  the  7-skill  level 
in  each  month/the  total  number  of  enlisted 
personnel  assigned  to  the  unit  during  the 
month. 

As  before,  these  indicators  constitute  incomplete  measures  of  the  OdT 
loads  borne  by  the  units  because  they  fail  to  reflect  informal  OJT 
activities  such  as  qualification  training,  and  they  fail  to  account 
for  any  progress  achieved  by  trainees  during  OJT. 

Next,  four  indicators  reflecting  the  availability  of  resources  to 
the  units  have  been  derived: 

•  LAMN:  shortages  in  the  assignment  of  enlisted  per¬ 

sonnel 

■  1.0  -  (the  number  of  enlisted  personnel 
assigned  to  the  unit  in  each  month/the  number 
of  enlisted  personnel  authorized  for  the  unit 
in  the  month). 

•  LSKILL:  shortages  in  the  availability  of  skilled  per¬ 

sonnel 

■  1.0  -  (the  average  skill  level  of  the  enlisted 
personnel  assigned  to  the  unit  in  each  month/the 
average  skill  level  of  the  enlisted  personnel 
authorized  for  the  unit  in  the  month). 

•  INEXP:  the  relative  inexperience  of  the  enlisted 

personnel 

*  the  number  of  3-skill  level  personnel  assigned 
to  the  unit  during  each  month/the  number  of  5-, 

7-,  and  9-skill  level  personnel  assigned  to  the 
unit  during  the  month. 

•  TRNRS:  the  availability  of  professional  training 

personnel 

■  the  number  of  personnel  with  PAFSC  75XXX 
assigned  to  the  unit  in  each  month/the  total 
number  of  enlisted  personnel  assigned  to  the 
unit  in  the  month. 

To  facilitate  direct  comparisons  with  the  training  load  indicators, 
the  first  three  resource  availability  indicators  have  been  designed  to 
describe  deficiencies  in  the  availability  of  resources  to  the  aircraft 
maintenance  units.  Only  TRNRS  represents  a  direct  measure  of  the  avail¬ 
ability  of  resources  to  the  units. 
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Finally,  to  reflect  key  aspects  of  the  effects  of  weather  condi¬ 
tions  on  mission  performance  and  training  quality,  a  dummy  variable 
distinguishing  the  cold  weather  months  from  the  other,  more  temperate 
months  has  been  included  as  an  indicator  of  external  factors.  This 
variable,  SEASON,  is  set  equal  to  one  for  each  month  between  November 
and  March,  and  to  zero  for  each  other  month,  to  compensate  for  the 
principal  seasonal  variations  in  the  level  and  quality  of  mission 
performance  and  the  quality  of  training. 

Due  primarily  to  the  large  number  of  indicators  of  the  quality 
of  mission  performance  identified  for  the  aircraft  maintenance  units, 
no  attempt  has  been  made,  during  the  empirical  testing  of  the  conceptual 
model  of  OJT  capacity  for  these  units,  to  investigate  the  direct  inter¬ 
actions  among  the  level  of  mission  performance,  the  quality  of  mission 
performance,  and  the  quality  of  training.  Consequently,  the  relation¬ 
ships  actually  examined  for  the  aircraft  maintenance  units  have  been 

ML  *  F  (TL,  R,  E) 

MQ  *  G  (TL,  R,  E) 

TQ  -  H  (TL,  R,  E) 

Moreover,  assuming  basic  linear  relationships,  the  general  forms  of  the 
equations  actually  estimated  for  these  units  are 


where 
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MQ  -  NGBORTS ,  NABORTS,  NBORTS ,  NMNTC,  NMATTR,  NFAIL, 

RR3HR,  RR6HR ,  ONTIME,  SORTYS,  MEFFC,  NNMCU, 
NNMCUS ,  NPMCM,  NPCNM,  NRCWU,  or  BREPR. 

TQ  -  QINSPC,  PINSPC ,  or  SAFET. 

TL  -  PIRN,  STRN,  and/or  RES 7. 

R  -  LAMN,  LSKILL,  INEXP,  and/or  TRNRS. 

E  -  SEASON. 


However,  inconsistencies  among  the  time  periods  for  which  different  data 
series  are  available,  combined  with  extensive  multicollinearity  among 
the  explanatory  variables,  have  precluded  examination  of  all  conceivable 
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formulations  of  the  three  basic  equations.  Nevertheless,  a  total  of 
91  different  formulations  of  the  basic  equations,  in  which  mission 
performance  level,  mission  performance  quality,  and  training  quality 
are  related  to  training  load,  resource  availability,  and. external 
factors,  have  been  estimated  for  the  four  aircraft  maintenance  units. 
These  formulations  all  describe  versions  of  the  general  equations  for 
which  all  estimated  coefficients  are  statistically  significant  at, 
at  least,  the  10  percent  level.  The  results  of  these  empirical  tests 
of  the  fundamental  components  of  the  conceptual  model  of  OJT  capacity 
are  discussed  in  the  following  three  subsections. 

5.3.1  Mission  Performance  Level 

Eight  of  the  91  empirical  relationships  estimated  for  the  aircraft 
maintenance  units  have  examined  the  association  between  the  level  of 
mission  performance  attained  by  a  unit  and  the  training  load,  resource 
availability,  and  external  factors  affecting  the  unit.  As  explained 
previously,  only  one  usable  indicator  of  the  level  of  mission  perfor¬ 
mance  attributable  to  the  aircraft  maintenance  units  has  been  obtained 
in  this  study.  The  empirical  results  derived  by  using  multiple  linear 
regression  analysis  to  develop  estimates  of  the  relationship  between 
this  indicator,  FLY HRS ,  and  the  various  indicators  of  training  load, 
resource  availability,  and  external  factors  are  presented  in  Table  8. 

These  results  fail  to  reveal  any  consistent  or  strong  association 
between  the  level  of  mission  performance  achieved  by  a  unit  and  either 
the  amount  of  training  provided  by  the  unit  or  —  within  the  range  of 
resource  allotments  observed  in  the  study  —  the  availability  of 
resources  to  the  unit.  In  only  two  of  the  estimated  equations  are 
training  load  variables  related  statistically  significantly  to  FLYHRS, 
and  in  one  of  these  instances,  the  sign  of  the  estimated  coefficient 
is  contrary  to  prior  expectations.  Similarly,  resource  availability 
variables  enter  only  three  of  the  equations  statistically  significantly, 
and  on  two  of  these  occasions  the  estimated  coefficients  have  unantici¬ 
pated  signs.  This  incidence  and  distribution  of  statistically  signif¬ 
icant  estimated  coefficients  can  reasonably  be  expected  to  occur 
randomly  in  the  absence  of  any  actual  association  between  the  dependent 
and  explanatory  variables.  Consequently,  the  empirical  results  provide 
no  evidence  of  any  interdependency  between  the  level  of  mission  per¬ 
formance,  training  load,  and  resource  availability  —  at  least  in  the 
low  stress,  peacetime  situations  observed  in  this  study. 

5.3.2  Mission  Performance  Quality 

A  total  of  73  equations  have  been  estimated  for  the  aircraft  main¬ 
tenance  units  to  investigate  the  relationships  between  each  of  the  17 
Indicators  of  mission  performance  quality  and  the  various  indicators 
of  training  loads,  resource  availability,  and  external  factors.  The 
detailed  empirical  results  of  these  Investigations  are  presented  in 
Tables  A-l  through  A-17  of  the  Appendix. 


These  results  are  generally  supportive  of  the  hypotheses  embodied 
in  the  conceptual  model  of  OJT  capacity  developed  in  Section  3.0.  How¬ 
ever,  this  support  is  not  unequivocal.  No  statistically  significant 
coefficients  have  been  estimated  for  any  of  the  training  load  or 
resoutce  availability  variables  in  15  of  the  74  equations.  Only  the 
external  factors  variable,  SEASON,  has  been  determined  to  be  a  statis¬ 
tically  significant  explanatory  variable  in  these  relationships. 
Moreover,  of  the  82  statistically  significant  coefficients  estimated 
for  training  load  and  resource  availability  variables  in  the  remaining 
58  equations,  26  have  signs  opposite  to  prior  expectations. 

Nevertheless,  a  substantial  majority  —  68  percent  —  of  these 
estimated  coefficients  exhibit  precisely  the  signs  anticipated  before 
the  empirical  analysis.  In  addition,  the  number  of  statistically 
significant  coefficients  estimated  for  the  training  load  and  resource 
availability  variables  is  considerably  larger  than  the  number  that 
would  be  expected  to  originate  through  purely  random  processes. 

Furthermore,  separate  examination  of  just  the  coefficients  esti¬ 
mated  for  the  training  load  variables  reveals  that  training  load  vari¬ 
ables  enter  20  of  the  estimated  equations  statistically  significantly 
—  and  that  in  18  of  these  equations  their  estimated  coefficients 
exhibit  the  anticipated  signs.  Thus,  on  only  two  occasions  is  the 
sign  of  an  estimated  coefficient  of  a  training  load  variable  inconsis¬ 
tent  with  prior  expectations.  These  results  are  clearly  indicative 
of  a  persistent  and  strong  inverse  relationship  between  the  amount 
of  training  conducted  in  a  unit  and  the  quality  of  mission  performance 
attributable  to  the  unit. 

Yet,  these  results  also  indicate  that  the  preponderance  of  the 
unanticipated  empirical  results  derived  for  the  aircraft  maintenance 
units  must  be  associated  with  the  resource  availability  variables. 
Nevertheless,  even  for  these  variables,  the  number  of  statistically 
significant  estimated  coefficients  with  signs  consistent  with  prior 
expectations  is  substantially  greater  than  the  number  of  such  estimates 
with  unanticipated  signs.  A  total  of  62  statistically  significant 
coefficients  have  been  estimated  for  resource  availability  variables 
in  49  of  the  equations  examining  mission  performance  quality.  Of 
these  62  coefficients,  38  have  the  anticipated  signs;  while  only  24 
have  signs  contrary  to  expectations.  With  63  percent  of  the  estimated 
coefficients  exhibiting  the  expected  signs,  the  empirical  results  are 
basically  supportive  of  the  presumed  relationships  between  resource 
availability  and  the  quality  of  mission  performance. 

Finally,  it  is  informative  to  examine  separately  the  relationships 
estimated  for  each  of  the  17  indicators  of  mission  performance  quality. 
Thus,  Table  9  presents,  for  each  mission  performance  quality  indicator, 
the  incremental  and  cumulative  frequency  distributions  of  the  signifi¬ 
cance  levels  of  the  regression  coefficients  estimated  for  the  training 
load  variables.  Similar  distributions  for  the  estimated  coefficients 
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of  the  resource  availability  variables  appear  in  Table  10.  Then, 
aggregate  distributions  pertaining  to  the  coefficients  estimated  for 
both  the  training  load  and  the  resource  availability  variables  are 
reported  in  Table  11. 

These  tables  clearly  reveal  that  considerable  differences  exist 
among  the  frequency  distributions  associated  with  the  various  mission 
performance  quality  indicators.  The  signs  of  the  coefficients  esti¬ 
mated  for  some  indicators  are  highly  consistent  with  prior  expectations 
for  both  the  training  load  variables  and  the  resource  availability 
variables.  For  other  indicators,  such  consistency  is  observed  only 
for  the  training  load  variables  or,  in  one  case,  only  for  the  resource 
availability  variables.  For  the  remaining  indicators,  substantial 
Inconsistency  with  prior  expectations  is  revealed  for  both  sets  of 
explanatory  variables. 

Some  appreciation  of  the  possible  origins  of  these  differences 
can  be  gained  by  considering  in  greater  detail  the  interpretations  of 
the  various  mission  performance  quality  indicators.  Each  of  the  17 
Indicators  captures  some  aspects  of  the  quality  of  mission  performance 
attributable  to  the  aircraft  maintenance  units.  Yet,  there  are  clear 
distinctions  among  the  perspectives  on  mission  performance  quality 
provided  by  the  different  indicators. 

Thus,  the  first  six  indicators  —  NGBORTS,  NABORTS,  NBORTS, 

NMNTC,  NMATTR ,  and  NFAIL  —  all  represent  measures  of  the  absolute 
quality  of  primary  mission  performance  achieved  by  the  flying  wings 
whose  aircraft  are  serviced  by  the  aircraft  maintenance  units.  For 
these  indicators,  in  general,  the  signs  of  the  coefficients  estimated 
for  both  the  training  load  and  the  resource  availability  variables 
exhibit  a  high  degree  of  consistency  with  prior  expectations.  Only 
the  indicator  NFAIL  reveals  any  notable  disagreement  with  anticipation, 
and  even  for  this  indicator,  a  majority  of  the  estimated  coefficients 
have  the  expected  signs. 

The  next  five  indicators  —  RR3HR,  RR6HR,  MEFFC,  ONTIME,  and 
SORTYS  —  appraise  the  ability  of  either  the  aircraft  maintenance  units 
themselves  or  else  the  flying  wings  whose  aircraft  are  maintained  on 
the  units,  to  fulfill  predetermined  performance  schedules  or  standards. 
The  signs  of  the  estimated  training  load  coefficients  associated  with 
these  Indicators  are  uniformly  identical  to  the  predicted  signs.  How¬ 
ever,  considerably  less  compatibility  with  prior  expectations  is 
observed  for  the  coefficients  of  the  resource  availability  variables. 
Only  for  RR3HR  and  RR6HR,  the  variables  most  clearly  and  directly 
related  to  quality  of  performance  of  maintenance  activities,  do  a 
majority  of  the  coefficients  estimated  for  the  resource  availability 
variables  have  the  anticipated  signs. 
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TABLE  10:  Distributions  of  Significance  Levels  of  Resource 
Availability  Coefficients  In  Mission  Performance 
Quality  Regressions  for  Aircraft  Maintenance  Units 


w 
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The  next  four  indicators  of  mission  performance  quality  —  NNMCU, 
NNMCUS,  NFMCM,  NPCNM  —  reflect  discretionary,  and  hence  to  some  extent 
arbitrary,  classifications  of  the  reasons  for,  and  severity  of,  the 
lack  of  full  mission  capability  of  aircraft  requiring  maintenance. 
Although  the  signs  of  the  training  load  coefficients  derived  for  these 
indicators  conform  reasonably  well  with  prior  expectations,  it  is  also 
true  that  the  only  two  statistically  significant  estimated  training 
load  coefficients  with  unanticipated  signs  appear  in  equations  pertain¬ 
ing  to  these  indicators.  Moreover,  the  resource  availability  coeffi¬ 
cients  estimated  for  these  indicators  typically  have  signs  opposite 
to  those  predicted. 

The  remaining  two  mission  performance  quality  indicators  —  NRCWU 
and  BREPR  —  represent  indirect,  and  somewhat  conceptually  questionable, 
measures  of  the  absolute  quality  of  maintenance  support  provided  by  the 
aircraft  maintenance  units.  Thus,  the  indicator  NRCWU  does  not  describe 
just  the  recurrence  for  some  aircraft  of  precisely  the  same  maintenance 
requirement  recently  treated  by  a  unit.  Rather,  it  also  reflects  the 
occurrence  in  close  succession  of  unrelated  maintenance  requirements 
within  the  same  aircraft  component.  Similarly,  the  indicator  BREPR 
not  only  measures  instances  in  which  aircraft  maintenance  units  are 
unable  to  accomplish  repairs  which  are  normally  performed  within  the 
units,  but  also  enumerates  situations  in  which,  as  a  matter  of  policy, 
units  never  perform  certain  repairs  internally.  Consequently,  it  is 
not  surprising  that,  for  these  two  indicators  of  mission  performance 
quality,  no  statistically  significant  relationships  are  derived  for 
the  training  load  variables,  and  a  substantial  portion  of  the  coeffi¬ 
cients  estimated  for  the  resource  availability  variables  have  unantic¬ 
ipated  signs. 

Thus,  in  summary,  the  empirical  results  derived  for  the  mission 
performance  quality  indicators  reveal 

•  A  persistent  and  strong  inverse  relationship  between 
the  amount  ■>{  training  conducted  within  aircraft 
maintenance  units  and  most  indicators  of  mission 
performance  quality. 

•  A  somewhat  weaker,  but  demonstrable,  association  in 
the  anticipated  direction  between  the  amount  of 
resources  available  to  the  units  and  the  quality  of 
mission  performance. 

In  addition,  these  results  are  demonstrated  most  strongly  for  the 
mission  performance  quality  indicators  that  directly  measure  the 
absolute  quality  of  primary  mission  performance  achieved  by  the  flying 
wings  whose  aircraft  are  serviced  by  the  aircraft  maintenance  units. 

The  more  equivocal  results  are  associated  with  the  more  questionable 
Indicators  of  mission  performance  quality. 


5.3.3  Training  Quality 


The  final  10  equations  estimated  for  the  aircraft  maintenance 
units  examine  the  relationship  between  the  quality  of  training  con¬ 
ducted  in  a  unit  and  the  training  load,  resource  availability,  and 
external  factors  affecting  the  unit.  The  detailed  empirical  results 
derived  in  these  examinations  for  the  three  indicators  of  training 
quality  developed  in  this  study  are  reported  in  Tables  A-18  through 
A- 20  of  the  Appendix.  In  addition,  Table  12  presents,  for  each 
training  quality  indicator,  the  incremental  and  cumulative  frequency 
distributions  of  the  significance  levels  of  the  regression  coeffi¬ 
cients  calculated  for  the  training  load  variables  alone,  the  resource 
availability  variables  alone,  and  both  of  these  sets  of  variables  in 
combination. 

Overall,  these  results  reveal  substantial  deviations  between  the 
signs  estimated  for  the  coefficients  of  these  variables  and  the  signs 
predicted  for  these  coefficients.  Only  one-third  of  the  estimated 
coefficients  exhibit  the  anticipated  signs. 

Yet,  as  with  the  indicators  of  mission  performance  quality, 
noticeable  differences  exist  among  the  results  derived  for  the  dif¬ 
ferent  training  quality  indicators.  For  QINSPC,  the  only  statistic¬ 
ally  significant  training  load  coefficient  has  the  expected  sign,  and 
the  majority  of  the  estimated  resource  availability  coefficients 
exhibit  signs  consistent  with  prior  expectations.  Conversely,  for 
PINSPC  and  SAFET,  the  signs  of  both  coefficients  calculated  for  train¬ 
ing  load  variables,  and  the  signs  of  virtually  all  coefficients  esti¬ 
mated  for  resource  availability  variables,  are  opposite  to  the 
predicted  signs. 

While  these  results  are  unanticipated,  a  reasonable  explanation 
for  their  occurrence  can  be  readily  deduced  by  examining  the  interpre¬ 
tations  of  the  three  training  quality  indicators  in  greater  depth. 

All  of  the  indicators  are  based  on  inspections  of  maintenance  tasks 
performed  not  only  by  recent  trainees,  but  also  by  experienced  enlisted 
personnel.  In  addition,  the  indicators  PINSPC  and  SAFET  reflect  the 
results  of  personnel  inspections  assessing  the  extent  to  which  per¬ 
sonnel  use  approved  procedures  when  performing  maintenance  tasks. 

Thus,  the  unexpected  signs  estimated  for  the  training  load  and 
resource  availability  coefficients  in  the  equations  pertaining  to 
PINSPC  and  SAFET  quite  possibly  result  from  the  application  of  uncon¬ 
ventional,  unapproved,  and  more  risky  maintenance  techniques  by  the 
more  experienced,  more  fully  trained  personnel. 

The  indicator  QINSPC,  in  contrast,  is  based  on  the  results  of 
quality  verification  inspections  evaluating  the  quality  of  the  repairs 
produced  through  the  performance  of  maintenance  tasks.  To  the  extent 
that  the  unconventional  maintenance  techniques  employed  by  experienced 
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personnel  are  quicker  or  more  effective  than  the  approved  procedures, 
the  estimated  training  load  and  resource  availability  coefficients  will 
exhibit  the  predicted  signs.  Conversely,  to  the  extent  that  the  uncon¬ 
ventional  maintenance  procedures  are  less  efficient  than. the  standard 
techniques,  the  signs  of  the  estimated  coefficients  will  be  contrary 
to  prior  expectations.  Therefore,  since  some  of  the  modified  mainte¬ 
nance  procedures  are  likely  to  be  more  efficient,  and  thers  will 
probably  be  less  efficient,  than  the  standard  techniques,  the  mixed 
results  derived  for  QINSPC  seem  reasonable. 

Finally,  it  should  be  remembered  that,  because  the  available 
indicators  of  training  quality  all  focus  on  post-training  task  per¬ 
formance,  it  is  equally  reasonable  to  interpret  the  indicators  as 
measures  of  the  quality  of  mission  performance.  This  unavoidable 
ambiguity  should  be  taken  into  account  when  assessing  the  empirical 
results  as  evidence  of  the  appropriateness  of  the  conceptual  model 
of  OJT  capacity. 

5.3.4  Summary 

The  empirical  results  derived  by  estimating  the  fundamental 
equations  of  the  conceptual  model  of  OJT  capacity  for  four  aircraft 
maintenance  units  indicate  that 

•  A  consistently  strong  inverse  relationship  exists 
between  the  amount  of  training  conducted  by  a  unit 
and  the  quality  of  mission  performance  attributable 
to  the  unit. 

•  A  perceptible,  but  weaker,  association  in  the  antic¬ 
ipated  direction  prevails  between  the  resources  avail¬ 
able  to  a  unit  and  the  quality  of  the  unit's  mission 
performance. 

•  The  most  robust  correlations  of  training  load  and 
resource  availability  with  the  quality  of  mission 
performance  are  observed  for  the  indicators  of  mission 
performance  quality  that  directly  measure  the  absolute 
quality  of  primary  mission  performance  achieved  by  the 
flying  wings  whose  aircraft  are  serviced  by  the  air¬ 
craft  maintenance  units. 

•  A  possible  association  exists  between  the  quality  of 
training  provided  in  a  unit  and  the  training  load  and, 
conceivably,  the  available  resources  of  the  unit.  The 
most  persuasive  evidence  supporting  this  relationship 
is  obtained  when  training  quality  is  measured  on  the 
basis  of  assessments  of  the  results  produced  through 
maintenance  activities.  However,  due  largely  to 


limited  data  availability  and  ambiguity  in  the  inter¬ 
pretation  of  the  obtainable  training  quality  indicators, 
the  existence  of  this  correlation  has  not  been  defini¬ 
tively  determined  in  this  study. 

•  At  least  in  the  relatively  low  stress,  peacetime  situ¬ 
ations  examined  in  the  study,  no  persistent  inter¬ 
dependency  is  discernible  among  the  level  of  mission 
performance  achieved,  the  amount  of  training  conducted, 
and  the  availability  of  resources  in  aircraft  mainte¬ 
nance  units. 

Thus,  overall,  the  empirical  analysis  of  the  aircraft  maintenance  units 
does  not  provide  unequivocal  support  for  each  of  the  fundamental  rela¬ 
tionships  embodied  in  the  conceptual  model  of  OJT  capacity  formulated  in 
Section  3.0.  Nevertheless,  particularly  when  the  limited  availability 
of  pertinent  data  and  the  ambiguous  interpretation  of  obtainable  data 
are  acknowledged,  the  results  of  the  analysis  provide  ample  support  for 
the  basic  principle  underlying  the  model  —  the  principle  that  the 
capacity  of  Air  Force  operational  units  to  conduct  OJT  is  not  unlimited. 
Rather,  it  is  ultimately  determined  by  the  mission  responsibility  and 
resource  endowments  of  the  units. 
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6.0  CONCLUSIONS  AND  RECOMMENDATIONS 


Despite  severe  limitations  in  the  availability  of  pertinent  data 
from  existing  sources,  a  persistent  statistically  significant  relation¬ 
ship  has  been  observed,  for  all  Air  Force  operational  units  studied, 
between  the  quality  of  mission  performance  achieved  by  a  unit  and  the 
amount  of  training  conducted  in  the  unit.  However,  the  analytic  results 
only  moderately  support  the  possibility  of  an  association  between  the 
quality  of  training  provided  in  a  unit  and  the  amount  of  training  per¬ 
formed  in  the  unit;  and  the  evidence  concerning  the  existence  of  inter¬ 
dependence  between  the  level  of  mission  performance  attained  by  a  unit 
and  the  training  load  of  the  unit  is  negligible  and  contradictory. 

Nevertheless,  especially  considering  the  relatively  poor  quality 
of  the  obtainable  data  and  the  absence  of  several  important  explanatory 
variables,  the  empirical  results  must  be  interpreted  as  extremely  favor¬ 
able  indications  that  an  operational  methodology  for  estimating  the 
capacity  of  Air  Force  operational  units  to  conduct  OJT  can  ultimately 
be  implemented.  Moreover,  the  results  strongly  attest  that  the  con¬ 
ceptual  model  of  OJT  capacity  formulated  and  preliminarily  tested  in 
this  study  can  appropriately  and  effectively  provide  the  fundamental 
premises  and  basic  analytic  structure  for  this  methodology. 

However,  the  development  of  an  implementable  methodology  will 
require  further  exploratory  research,  supported  by  a  systematic  pro¬ 
gram  of  data  collection  and  retention  designed  to  provide  a  compre¬ 
hensive  set  of  indicators  describing  all  of  the  variables  contained 
In  the  conceptual  model.  The  remainder  of  this  section  presents  spe¬ 
cific  recommendations  concerning  prospective  data  acquisition  and 
research  activities. 


6. 1  Darn  Acquisition  Recommendations 


The  validation  and  calibration  of  a  model  of  OJT  capacity  requires 
historical  data.  Specifically,  for  the  model  formulated  in  this  report, 
data  suitable  for  creating  six  different  sets  of  indicators  must  be 
acquired.  The  types  of  indicators  to  be  created  include 

•  Mission  performance  level  indicators,  describing  the 
actions  performed  by  Air  Force  operational  units,  and 
specifying  the  magnitudes  of  these  actions. 

•  Mission  performance  quality  indicators,  assessing  the 
quality  of  the  outputs  directly  or  ultimately  produced 
through  the  units'  actions.  Normally,  these  indicators 
will  be  based  on  standard  quality  control  measures, 
such  as  tabulations  of  the  number  of  errors  committed 
or  the  speed  of  task  performance. 


59 

. . . . . £ 


•  Training  load  Indicators,  reporting  the  total  quantity 
of  training  conducted  within  the  units,  including  both 
currently  recorded  UGT  enrollment  and  informal  activ¬ 
ities  such  as  qualification  training.  In  addition, 
some  assessment  of  trainees'  progress  in  OJT  would  be 
desirable. 

•  Training  quality  indicators,  evaluating  the  quality  of 
the  training  conducted  in  the  units,  preferably  in  terms 
directly  relevant  to  the  units'  mission  responsibilities. 

•  Resource  availability  indicators,  characterizing  the 
resources  allocated  to  the  units  for  mission  performance 
or  training.  The  resource  endowments  described  should 
include  personnel,  equipment,  and  financial  assets. 

Moreover,  if  possible,  measurements  of  personnel 
resources  should  be  partitioned  by  skill. 

•  External  factors  indicators,  describing  phenomena  beyond 
control  of  the  units  that  might  affect  their  training  or 
mission  performance.  Pertinent  indicators  for  many  units 
might  include  weather  conditions,  or  the  diversion  of 
personnel  to  activities  outside  the  units. 

The  results  derived  in  the  current  study  clearly  demonstrate  that, 
for  different  types  of  operational  units,  the  precise  forms  of  the  pro¬ 
spectively  pertinent  indicators  for  each  of  the  sets  listed  above  will 
be  different.  Therefore,  to  determine  the  particular  data  items  that 
must  be  acquired  to  create  such  indicators,  it  will  first  be  necessary 
to  select  the  specific  types  of  operational  units  for  which  the  con¬ 
ceptual  model  of  OJT  capacity  will  be  empirically  analyzed  in  the  next 
research  effort. 

Then,  it  will  be  necessary  to  develop  precise  definitions  of  the 
indicators  prospectively  pertinent  to  each  of  the  selected  types  of 
units.  The  definitions  should  reflect  all  aspects  of  each  set  of 
indicators  believed  to  affect  the  units,  but  should  be  susceptible 
to  as  few  extraneous  influences  or  contradictory  interpretations  as 
possible. 

Next,  the  definitions  should  be  decomposed  to  isolate  the  indi¬ 
vidual  component  data  items  from  which  the  indicators  are  formed. 

Thus,  for  example,  ratios  should  be  split  into  their  numerators  and 
denominators,  and  measures  of  change  over  time  should  be  expressed 
in  terms  of  their  underlying  data  series. 

Whenever  possible,  to  minimize  the  expenditure  of  time  and  effort, 
the  resultant  desired  data  elements  should  then  be  obtained  from  estab¬ 
lished  sources.  In  many  instances,  the  desired  data  will  correspond  to 
Information  already  routinely  collected  and  maintained  by  the  Air  Force. 
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However,  in  ocher  cases,  Che  daca,  while  regularly  collected  by 
Che  Air  Force,  may  noC  be  reCained  for  a  long  enough  period  of  cime, 
or  may  noC  be  preserved  in  a  suitable  form,  to  be  readily  usable  for 
the  proposed  analysis.  For  example,  some  information  may  currently 
be  compiled  only  for  the  squadron  or  the  flying  wing,  while  tabulations 
for  individual  work  centers  might  be  needed  if  the  interactions  among 
resources,  training  loads,  and  mission  performance  that,  in  combination, 
determine  OJT  capacity  are  to  be  thoroughly  investigated.  In  these 
situations,  revised  data  recording  amd  retention  procedures  should  be 
established  to  assure  the  availability  of  the  required  information. 

Finally,  some  desired  data  elements  may  not  presently  be  collected 
in  any  form  by  any  identifiable  agency.  For  these  data  elements,  com¬ 
pletely  new  data  acquisition  and  retention  procedures  must  be  estab¬ 
lished  and  implemented. 

However,  for  purposes  of  model  validation  and  calibration,  the 
data  acquisition  activities  outlined  previously  need  not  be  undertaken 
universally  throughout  the  Air  Force.  Rather,  a  restricted  number  of 
operational  units  of  each  designated  type  can  be  selected  to  serve  as 
representative  prototypes  for  their  general  class  of  units.  The  estab¬ 
lished  data  collection,  compilation,  and  retention  procedures  can  then 
be  applied  exclusively  to  the  selected  units.  Only  after  the  OJT 
Capacity  estimation  methodology  has  been  fully  demonstrated,  and  the 
precise  data  items  applicable  to  this  methodology  have  been  determined, 
should  expanded  data  acquisition  procedures  be  implemented  in  all  oper¬ 
ational  units. 


6. 2  Data  Analysis  Recommendations 


The  encouraging  results  derived  in  this  study  suggest  that  further 
research  efforts  to  validate  and  calibrate  the  conceptual  model  of  OJT 
capacity  on  the  basis  of  improved  data  may  ultimately  prove  to  be  highly 
beneficial  to  the  Air  Force.  Specifically,  using  the  information 
obtained  through  the  data  acquisition  efforts  outlined  in  the  preceding 
section,  empirical  estimates  of  the  coefficients  of  each  of  the  model's 
basic  equations  should  be  derived.  Essentially,  this  involves  the  esti¬ 
mation  of  production  functions  describing  the  level  and  quality  of 
mission  performance  and  training  provided  in  individual  operational 
units.  Other  studies  have  attested  to  the  difficulty  of  this  task. ^ 
However,  the  results  produced  in  this  study  indicate  that  such  relation¬ 
ships  can  successfully  be  established,  at  least  for  the  factors  most 
pertinent  to  estimating  OJT  capacity,  and  with  sufficient  accuracy  to 
permit  the  derivation  of  satisfactory  capacity  estimates. 


See,  for  example,  Z.  Barzily,  W.  H.  Marlow,  and  S.  Zacks,  "Survey 
of  Approaches  to  Readiness,"  Naval  Research  Logistics  Quarterly, 
Arlington,  VirgLnia,  Office  of  Naval  Research,  March  1979,  pp.  21-31. 


The  empirical  analysis  to  be  undertaken  in  the  next  research 
effort  can  be  conducted  in  either  of  two  ways.  First,  if  the  deriva¬ 
tion  of  improved  empirical  results  is  desired  as  quickly  as  possible, 
the  equations  in  the  conceptual  model  can  be  estimated  using  cross- 
sectional  data  describing  conditions  in  a  relatively  large  number  of 
similar  operational  units  during  a  relatively  short  period  of  time. 
However,  this  approach  implicitly  assumes  that  the  functional  rela¬ 
tionships  determining  OJT  and  mission  performance  are  identical  in 
all  of  these  units,  but  typically  does  not  provide  sufficient  data 
to  test  the  validity  of  this  assumption. 

Alternatively,  the  equations  in  the  conceptual  model  can  be  esti¬ 
mated  using  time  series  data  for  a  smaller  number  of  operational  units, 
but  for  a  longer  period  of  time.  Obviously,  adoption  of  this  approach 
will  necessarily  delay  the  derivation  of  improved  analytic  results. 
However,  it  will  also  permit  the  direct  comparison  of  the  estimated 
equations  derived  for  similar  operational  units  to  test  the  validity 
of  the  assumption  that  the  corresponding  functional  relationships  are 
identical. 

If  this  assumption  is  confirmed  for  any  of  the  sets  of  operational 
units  examined,  a  satisfactory  OJT  capacity  estimation  methodology  could 
be  directly  established  for  these  units  on  the  basis  of  the  estimated 
equations.  Conversely,  if  the  empirical  results  reveal  differences 
among  the  estimated  equations  derived  for  presumably  similar  units, 
an  investigation  should  be  undertaken  to  determine  the  sources  of  the 
observed  variability.  In  general,  this  would  entail  refinement  of  the 
conceptual  model,  possibly  through  the  inclusion  of  additional  explan¬ 
atory  variables  or  through  respecification  of  the  presumed  relationships 
among  variables,  to  develop  a  formulation  that  accounts  for  the  observed 
differences. 

Only  if  this  effort  is  unsuccessful  for  some  type  of  operational 
units  will  implementation  of  an  OJT  capacity  estimation  methodology 
applicable  throughout  the  entire  Air  Force  entail  separate  estimation 
of  the  basic  equations  of  the  conceptual  model  of  OJT  capacity  for  each 
Individual  unit  of  that  type.  Moreover,  even  this  task,  while  obvi¬ 
ously  onerous  and  costly,  is  feasible  and  potentially  beneficial. 

In  particular,  the  derivation  of  satisfactory  estimates  of  the 
fundamental  equations  of  the  conceptual  model  in  the  manner  discussed 
above  will  permit  the  calculation  of  theoretically  sound  estimates  of 
the  capacity  of  operational  units  to  conduct  OJT.  Development  of  such 
OJT  capacity  estimates  will,  in  general,  involve  the  application  of 
mathematical  optimization  techniques  such  as  linear  programming  to 
derive  solutions  to  constrained  maximization  problems  such  as  those 
specified,  and  illustrated,  in  Section  3.0.  No  calculations  of  this 
type  have  been  performed  in  this  study  because  of  the  severe  data 
limitations  confronted  for  the  operational  units  examined.  Any  assess¬ 
ments  of  OJT  capacity  derived  under  these  circumstances  would  have  been 
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inconsequential  and  meaningless.  However,  improved  data  and  refined 
estimates  of  the  conceptual  model  of  OJT  capacity  can  reasonably  be 
expected  to  result  in  an  informative  and  useful  analysis. 

Finally,  the  availability  of  acceptable  estimates  of  the  basic 
equations  of  the  conceptual  model  of  OJT  capacity  for  a  substantial 
number  of  operational  units  will  permit  the  formulation  and  testing 
of  procedures  for  estimating  the  OJT  capacities  of  operational  units 
at  higher  organizational  levels  by  appropriately  aggregating  the 
capacities  estimated  for  lower- level  component  units.  At  the  extreme, 
this  might  entail  the  development  of  methodologies,  applicable  at  the 
base  or  MAJCOM  (Major  Command)  levels,  that  use  decomposition  algo¬ 
rithms  to  estimate  the  OJT  capacities  of  the  component  units.  More 
realistically,  it  should  be  possible  to  account  for  the  interdepend¬ 
encies  between  higher  level  and  lower  level  units  by  developing  func¬ 
tional  procedures  that  represent  simple  extensions  of  the  OJT  capacity 
estimation  procedures  applicable  to  the  component  units.  For  example, 
it  may  be  possible  to  establish  factors  denoting  base  and  MAJCOM 
capacity  constraints  that  can  be  included  directly  in  the  OJT  capacity 
estimation  methodologies  of  the  lower  level  units.  In  any  event,  the 
development  of  satisfactory  aggregation  procedures  will  permit  the 
estimation  of  OJT  capacities  for  virtually  any  organizational  level 
within  the  .Air  Force. 
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TABLE  A-2:  Empirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  NABORTS  as  Dependent  Variable 


TABLE  A-5:  Empirical  Results  of  Aircraft  Maintena.s'-c  Unit 
Regressions  with  NMATTR  as  Dependent  Variable 


Empirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  NFAIL  as  Dependent  Variable 


TABLE  A-8:  Bnpirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  RR6HR  as  Dependent  Variable 
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TABLE  A-10:  Empirical  Results  of  Aircraft  Maintenance  Unit 
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TABLE  A-14:  Empirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  NPMCM  as  Dependent  Variable 


Empirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  NRCWU  as  Dependent  Variable 


Empirical  ResulCs  of  Aircraft  Maintenance  Unit 
Regressions  with  PINSPC  as  Dependent  Variable 


TABLE  A-20:  Bnpirical  Results  of  Aircraft  Maintenance  Unit 
Regressions  with  SAFET  as  Dependent  Variable 
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